[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:12 dist-test-slave-rwrg krb5kdc[3028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:12 dist-test-slave-rwrg krb5kdc[3028](info): set up 2 sockets
Aug 07 06:33:12 dist-test-slave-rwrg krb5kdc[3028](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 06:33:13.779783 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:13 dist-test-slave-rwrg krb5kdc[3028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:13.903609 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:37929
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:37929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:13.986595 3048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:13.986830 3048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:13.986863 3048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:13.987968 3048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:13.988030 3048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:13.988054 3048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:13.988076 3048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:13.988111 3048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:13.989768 3048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:37929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:37929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:13.990144 3048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:13.990398 3048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:13.996376 3065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:13.996414 3059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:13.996676 3048 server_base.cc:1034] running on GCE node
W20260807 06:33:13.997329 3061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:13.997637 3048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:13.997938 3048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:13.999152 3048 hybrid_clock.cc:648] HybridClock initialized: now 1786084393999117 us; error 60 us; skew 500 ppm
Aug 07 06:33:13 dist-test-slave-rwrg krb5kdc[3028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084393, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:14.001761 3048 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:14.002427 3048 webserver.cc:460] Webserver started at http://127.2.243.62:33805/ using document root <none> and password file <none>
I20260807 06:33:14.002744 3048 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:14.002815 3048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:14.003003 3048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:14.004170 3048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "85360d4c5bdb418dae265a917e70e10f"
format_stamp: "Formatted at 2026-08-07 06:33:13 on dist-test-slave-rwrg"
server_key: "f6d5aa1db37d05ccb075c9cbad915ff5"
server_key_iv: "f4920bddd5c8e685990fd8f34fec2e2b"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.004567 3048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "85360d4c5bdb418dae265a917e70e10f"
format_stamp: "Formatted at 2026-08-07 06:33:13 on dist-test-slave-rwrg"
server_key: "f6d5aa1db37d05ccb075c9cbad915ff5"
server_key_iv: "f4920bddd5c8e685990fd8f34fec2e2b"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.006333 3048 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:14.007421 3073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.007741 3048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:14.007843 3048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "85360d4c5bdb418dae265a917e70e10f"
format_stamp: "Formatted at 2026-08-07 06:33:13 on dist-test-slave-rwrg"
server_key: "f6d5aa1db37d05ccb075c9cbad915ff5"
server_key_iv: "f4920bddd5c8e685990fd8f34fec2e2b"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.007913 3048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:14.019142 3048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:14.021353 3048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:14.021555 3048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:14.027033 3048 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:37929
I20260807 06:33:14.027101 3169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:37929 every 8 connection(s)
I20260807 06:33:14.027750 3048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:14.028787 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3048
I20260807 06:33:14.028931 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:14.029500 3171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:14.029954 3020 external_mini_cluster.cc:1428] Setting key dcff803799572fe69a5fe3e187bb75df
I20260807 06:33:14.033368 3171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f: Bootstrap starting.
I20260807 06:33:14.034469 3171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:14.034857 3171 log.cc:824] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:14.035635 3171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f: No bootstrap required, opened a new log
I20260807 06:33:14.038585 3171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85360d4c5bdb418dae265a917e70e10f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 37929 } }
I20260807 06:33:14.038791 3171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:14.038906 3171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85360d4c5bdb418dae265a917e70e10f, State: Initialized, Role: FOLLOWER
I20260807 06:33:14.039084 3171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85360d4c5bdb418dae265a917e70e10f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 37929 } }
I20260807 06:33:14.039258 3171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:14.039309 3171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:14.039361 3171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:14.040123 3171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85360d4c5bdb418dae265a917e70e10f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 37929 } }
I20260807 06:33:14.040351 3171 leader_election.cc:302] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85360d4c5bdb418dae265a917e70e10f; no voters:
I20260807 06:33:14.040645 3171 leader_election.cc:288] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:14.040707 3180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:14.040923 3180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [term 1 LEADER]: Becoming Leader. State: Replica: 85360d4c5bdb418dae265a917e70e10f, State: Running, Role: LEADER
I20260807 06:33:14.041158 3180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85360d4c5bdb418dae265a917e70e10f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 37929 } }
I20260807 06:33:14.041308 3171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:14.041785 3181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85360d4c5bdb418dae265a917e70e10f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85360d4c5bdb418dae265a917e70e10f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 37929 } } }
I20260807 06:33:14.041869 3181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:14.042037 3182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85360d4c5bdb418dae265a917e70e10f. Latest consensus state: current_term: 1 leader_uuid: "85360d4c5bdb418dae265a917e70e10f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85360d4c5bdb418dae265a917e70e10f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 37929 } } }
I20260807 06:33:14.042110 3182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:14.042330 3190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:14.043702 3190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:14.046072 3190 catalog_manager.cc:1347] Generated new cluster ID: 3a24f3a590384889b4d5d44377f637f3
I20260807 06:33:14.046129 3190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:14.049412 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.031321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60288 (local address 127.2.243.62:37929)
0807 06:33:14.031636 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:14.031647 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:14.031824 (+ 177us) server_negotiation.cc:407] Connection header received
0807 06:33:14.034270 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:14.034279 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:14.034392 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:14.034568 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:14.036221 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.037754 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.038821 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.039509 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.039649 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.044668 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:14.044672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:14.044672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:14.044684 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:14.047246 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.047747 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.047750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.047751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.047860 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.048154 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.048155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.048155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.048480 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:14.048704 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:14.048938 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 06:33:14.049145 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260807 06:33:14.062228 3190 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:14.062772 3190 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:14.067760 3190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85360d4c5bdb418dae265a917e70e10f: Generated new TSK 0
I20260807 06:33:14.067934 3190 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:14.095549 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:37929
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:14.178011 3214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:14.178243 3214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:14.178283 3214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:14.179373 3214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:14.179433 3214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:14.179479 3214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:14.181007 3214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:37929
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:14.181337 3214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:14.181555 3214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:14.187796 3226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:14.187796 3229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:14.187913 3225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:14.188139 3214 server_base.cc:1034] running on GCE node
I20260807 06:33:14.188356 3214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:14.188642 3214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:14.189795 3214 hybrid_clock.cc:648] HybridClock initialized: now 1786084394189778 us; error 33 us; skew 500 ppm
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:14.192284 3214 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:14.192830 3214 webserver.cc:460] Webserver started at http://127.2.243.1:34531/ using document root <none> and password file <none>
I20260807 06:33:14.193073 3214 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:14.193118 3214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:14.193249 3214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:14.194298 3214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "badad91b525341e6b39bea085cb52c0c"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "775bd42170de37824b1a1f2715233450"
server_key_iv: "12524f8f703062918d62cfa60b257e64"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.194660 3214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "badad91b525341e6b39bea085cb52c0c"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "775bd42170de37824b1a1f2715233450"
server_key_iv: "12524f8f703062918d62cfa60b257e64"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.196101 3214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:14.196910 3239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.197058 3214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.197144 3214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "badad91b525341e6b39bea085cb52c0c"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "775bd42170de37824b1a1f2715233450"
server_key_iv: "12524f8f703062918d62cfa60b257e64"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.197213 3214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:14.217247 3214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:14.219530 3214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:14.219696 3214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:14.219985 3214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:14.220355 3214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:14.220419 3214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.220444 3214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:14.220485 3214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.227223 3214 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:41313
I20260807 06:33:14.227274 3422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:41313 every 8 connection(s)
I20260807 06:33:14.227583 3214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:14.231281 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3214
I20260807 06:33:14.231388 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:14.231541 3020 external_mini_cluster.cc:1428] Setting key 5d71fe0b5af41da86130350d3f091e7a
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:14.237769 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.228542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:37929 (local address 127.2.243.1:58209)
0807 06:33:14.228754 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:14.228791 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:14.229272 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:14.229471 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:14.229474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:14.230230 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:14.230420 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:14.230425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.231095 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.231095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:14.232068 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:14.232070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.232484 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.232485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:14.232627 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.233112 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:14.233120 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:14.234582 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:14.236375 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:14.236381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:14.236383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:14.236674 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:14.236898 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:14.236899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:14.236899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:14.236998 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:14.237303 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:14.237307 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:14.237414 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 06:33:14.237486 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:33:14.237616 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
I20260807 06:33:14.237784 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.228625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:58209 (local address 127.2.243.62:37929)
0807 06:33:14.228683 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:14.228685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:14.228876 (+ 191us) server_negotiation.cc:407] Connection header received
0807 06:33:14.229354 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:14.229356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:14.229366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:14.229420 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:14.230500 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.231017 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.232158 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.232406 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.232478 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.234666 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:14.234668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:14.234668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:14.234677 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:14.236300 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.236753 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.236755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.236755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.236821 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.237090 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.237091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.237092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.237242 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:14.237317 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:14.237551 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 06:33:14.237647 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:33:14.238340 3423 heartbeater.cc:347] Connected to a master server at 127.2.243.62:37929
I20260807 06:33:14.238451 3423 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:14.238654 3423 heartbeater.cc:511] Master 127.2.243.62:37929 requested a full tablet report, sending...
I20260807 06:33:14.239558 3104 ts_manager.cc:194] Registered new tserver with Master: badad91b525341e6b39bea085cb52c0c (127.2.243.1:41313)
I20260807 06:33:14.240329 3104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:58209
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:14.259366 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:37929
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:14.341758 3433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:14.341975 3433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:14.342002 3433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:14.343204 3433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:14.343251 3433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:14.343278 3433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:14.344836 3433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:37929
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:14.345041 3433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:14.345216 3433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:14.350086 3442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:14.350122 3447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:14.350107 3445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:14.350239 3433 server_base.cc:1034] running on GCE node
I20260807 06:33:14.350564 3433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:14.350812 3433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:14.351986 3433 hybrid_clock.cc:648] HybridClock initialized: now 1786084394351964 us; error 34 us; skew 500 ppm
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:14.354547 3433 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:14.355080 3433 webserver.cc:460] Webserver started at http://127.2.243.2:40373/ using document root <none> and password file <none>
I20260807 06:33:14.355348 3433 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:14.355410 3433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:14.355544 3433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:14.356693 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "c2046d2e50294c60bf6916e6be8a032a"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "b441d18f4404175425935999be9c1902"
server_key_iv: "ee952eaf211699a4037fddd13048e929"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.357054 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "c2046d2e50294c60bf6916e6be8a032a"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "b441d18f4404175425935999be9c1902"
server_key_iv: "ee952eaf211699a4037fddd13048e929"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.358467 3433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:14.359344 3456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.359514 3433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:14.359637 3433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "c2046d2e50294c60bf6916e6be8a032a"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "b441d18f4404175425935999be9c1902"
server_key_iv: "ee952eaf211699a4037fddd13048e929"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.359719 3433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:14.385135 3433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:14.387558 3433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:14.387722 3433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:14.387976 3433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:14.388298 3433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:14.388351 3433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.388393 3433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:14.388425 3433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.395028 3433 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:37541
I20260807 06:33:14.395078 3651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:37541 every 8 connection(s)
I20260807 06:33:14.395416 3433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:14.404718 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3433
I20260807 06:33:14.404824 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:14.405002 3020 external_mini_cluster.cc:1428] Setting key 9e6bfba56e2e3d7e0fb973b394b63328
I20260807 06:33:14.405491 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.396193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:45019 (local address 127.2.243.62:37929)
0807 06:33:14.396253 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:14.396256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:14.396457 (+ 201us) server_negotiation.cc:407] Connection header received
0807 06:33:14.397034 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:14.397036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:14.397046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:14.397094 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:14.398096 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.398616 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.399576 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.399810 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.399868 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.402437 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:14.402439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:14.402439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:14.402450 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:14.404061 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.404427 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.404428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.404429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.404510 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.404841 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.404843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.404843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.404982 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:14.405060 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:14.405341 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 06:33:14.405424 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 06:33:14.405678 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.396150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:37929 (local address 127.2.243.2:45019)
0807 06:33:14.396315 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:14.396356 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:14.396918 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:14.397182 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:14.397185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:14.397793 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:14.397990 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:14.397995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.398706 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.398707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:14.399504 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:14.399505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.399879 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.399880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:14.399993 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.400531 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:14.400541 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:14.402330 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:14.404148 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:14.404154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:14.404155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:14.404361 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:14.404568 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:14.404569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:14.404570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:14.404721 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:14.405051 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:14.405057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:14.405209 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 06:33:14.405312 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 06:33:14.405475 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260807 06:33:14.406085 3654 heartbeater.cc:347] Connected to a master server at 127.2.243.62:37929
I20260807 06:33:14.406203 3654 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:14.406388 3654 heartbeater.cc:511] Master 127.2.243.62:37929 requested a full tablet report, sending...
I20260807 06:33:14.406833 3104 ts_manager.cc:194] Registered new tserver with Master: c2046d2e50294c60bf6916e6be8a032a (127.2.243.2:37541)
I20260807 06:33:14.407320 3104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:45019
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:14.431131 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:37929
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:14.515079 3663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:14.515282 3663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:14.515306 3663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:14.516552 3663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:14.516594 3663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:14.516623 3663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:14.518097 3663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:37929
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:14.518340 3663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:14.518606 3663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:14.523622 3669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:14.523627 3670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:14.523653 3672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:14.523903 3663 server_base.cc:1034] running on GCE node
I20260807 06:33:14.524117 3663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:14.524354 3663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:14.525497 3663 hybrid_clock.cc:648] HybridClock initialized: now 1786084394525487 us; error 35 us; skew 500 ppm
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:14.527987 3663 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:14.528424 3663 webserver.cc:460] Webserver started at http://127.2.243.3:38463/ using document root <none> and password file <none>
I20260807 06:33:14.528666 3663 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:14.528713 3663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:14.528859 3663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:14.529960 3663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "7b2ea8e4afb841fdace30dbad08f36d4"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "32b7d3d4e2b651d6242c6cf4728bba4d"
server_key_iv: "eda0270e5486fe942718d99dbd6ecb38"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.530329 3663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "7b2ea8e4afb841fdace30dbad08f36d4"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "32b7d3d4e2b651d6242c6cf4728bba4d"
server_key_iv: "eda0270e5486fe942718d99dbd6ecb38"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.531749 3663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:14.532477 3683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.532675 3663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:14.532765 3663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "7b2ea8e4afb841fdace30dbad08f36d4"
format_stamp: "Formatted at 2026-08-07 06:33:14 on dist-test-slave-rwrg"
server_key: "32b7d3d4e2b651d6242c6cf4728bba4d"
server_key_iv: "eda0270e5486fe942718d99dbd6ecb38"
server_key_version: "encryptionkey@0"
I20260807 06:33:14.532835 3663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:14.545604 3663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:14.547972 3663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:14.548133 3663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:14.548370 3663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:14.548765 3663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:14.548803 3663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.548825 3663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:14.548838 3663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:14.555302 3663 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:39263
I20260807 06:33:14.555380 3884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:39263 every 8 connection(s)
I20260807 06:33:14.555711 3663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:14.556316 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3663
I20260807 06:33:14.556416 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:14.556684 3020 external_mini_cluster.cc:1428] Setting key 189df9fec89c7bfc0e0646de58a19067
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:14.565043 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.556529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:37321 (local address 127.2.243.62:37929)
0807 06:33:14.556573 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:14.556576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:14.556818 (+ 242us) server_negotiation.cc:407] Connection header received
0807 06:33:14.557542 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:14.557543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:14.557555 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:14.557628 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:14.558594 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.559087 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.560084 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.560260 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.560311 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.562556 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:14.562557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:14.562558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:14.562564 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:14.563812 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.564201 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.564202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.564202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.564258 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.564471 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.564472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.564472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.564625 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:14.564679 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:14.564907 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 06:33:14.564969 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:14.565225 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.556524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:37929 (local address 127.2.243.3:37321)
0807 06:33:14.556692 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:14.556741 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:14.557395 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:14.557708 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:14.557712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:14.558302 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:14.558485 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:14.558490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.559164 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.559165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:14.560014 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:14.560016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.560328 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.560329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:14.560423 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.561035 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:14.561043 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:14.562457 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:14.563906 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:14.563911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:14.563913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:14.564134 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:14.564318 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:14.564319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:14.564319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:14.564417 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:14.564682 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:14.564686 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:14.564797 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 06:33:14.564882 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 06:33:14.564997 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260807 06:33:14.565698 3885 heartbeater.cc:347] Connected to a master server at 127.2.243.62:37929
I20260807 06:33:14.565800 3885 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:14.565935 3885 heartbeater.cc:511] Master 127.2.243.62:37929 requested a full tablet report, sending...
I20260807 06:33:14.566282 3104 ts_manager.cc:194] Registered new tserver with Master: 7b2ea8e4afb841fdace30dbad08f36d4 (127.2.243.3:39263)
I20260807 06:33:14.566659 3104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:37321
I20260807 06:33:14.568552 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:14.592926 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.584607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.2.243.1:41313)
0807 06:33:14.584816 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:14.584821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:14.584832 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:14.584933 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:14.584939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:14.584955 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:14.585055 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:14.585593 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.586290 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.587139 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.587482 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.587714 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.590216 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:14.590223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:14.590223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:14.590229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:14.591770 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.592101 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.592103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.592103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.592157 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.592320 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.592321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.592321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.592444 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:14.592676 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:14.592715 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 06:33:14.592776 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":121,"threads_started":1}
W20260807 06:33:14.593415 3286 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59410
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:14.617038 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:14.609731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.2.243.1:41313)
0807 06:33:14.609823 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:14.609826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:14.609941 (+ 115us) server_negotiation.cc:407] Connection header received
0807 06:33:14.610025 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:14.610026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:14.610043 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:14.610105 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:14.610763 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.611248 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.612077 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:14.612277 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:14.612347 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:14.614610 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:14.614612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:14.614612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:14.614619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:14.615881 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.616208 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.616209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.616209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.616280 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:14.616480 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:14.616481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:14.616481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:14.616616 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:14.616738 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:14.616838 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 06:33:14.616921 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:33:14.617550 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3214
I20260807 06:33:14.632339 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3433
I20260807 06:33:14.649633 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3663
I20260807 06:33:14.660081 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3048
2026-08-07T06:33:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2190 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3905](info): set up 2 sockets
Aug 07 06:33:14 dist-test-slave-rwrg krb5kdc[3905](info): commencing operation
krb5kdc: starting...
W20260807 06:33:15.929795 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:16.047981 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42265
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:16.129581 3924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:16.129779 3924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:16.129801 3924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:16.130934 3924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:16.130978 3924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:16.130992 3924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:16.131024 3924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:16.131075 3924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:16.132553 3924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:16.132793 3924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:16.133051 3924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:16.137893 3932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:16.138051 3924 server_base.cc:1034] running on GCE node
W20260807 06:33:16.137892 3931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:16.137887 3935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:16.138373 3924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:16.138679 3924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:16.139845 3924 hybrid_clock.cc:648] HybridClock initialized: now 1786084396139832 us; error 30 us; skew 500 ppm
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:16.145941 3924 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:16.146409 3924 webserver.cc:460] Webserver started at http://127.2.243.62:40677/ using document root <none> and password file <none>
I20260807 06:33:16.146672 3924 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:16.146739 3924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:16.146881 3924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:16.148125 3924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "221ecd53825f4eaa9a2b5dc60db2f137"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "320a3a6d3c2eae162be6d7e2f5227f51"
server_key_iv: "8193e43e4b9ce152d9176d257d10b1f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.148600 3924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "221ecd53825f4eaa9a2b5dc60db2f137"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "320a3a6d3c2eae162be6d7e2f5227f51"
server_key_iv: "8193e43e4b9ce152d9176d257d10b1f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.150024 3924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:16.150854 3947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.151028 3924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:16.151130 3924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "221ecd53825f4eaa9a2b5dc60db2f137"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "320a3a6d3c2eae162be6d7e2f5227f51"
server_key_iv: "8193e43e4b9ce152d9176d257d10b1f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.151207 3924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:16.160786 3924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:16.162556 3924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:16.162714 3924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:16.166549 3924 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42265
I20260807 06:33:16.166584 4028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42265 every 8 connection(s)
I20260807 06:33:16.166930 3924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:16.167753 4029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:16.169976 4029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137: Bootstrap starting.
I20260807 06:33:16.170504 4029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:16.170733 4029 log.cc:824] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:16.171378 4029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137: No bootstrap required, opened a new log
I20260807 06:33:16.172961 4029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42265 } }
I20260807 06:33:16.173123 4029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:16.173146 4029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221ecd53825f4eaa9a2b5dc60db2f137, State: Initialized, Role: FOLLOWER
I20260807 06:33:16.173274 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3924
I20260807 06:33:16.173388 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:16.173336 4029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42265 } }
I20260807 06:33:16.173579 3020 external_mini_cluster.cc:1428] Setting key 182010471604843c01ccfdc8df08557b
I20260807 06:33:16.173729 4033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:16.173813 4033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42265 } }
I20260807 06:33:16.173943 4029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:16.173919 4033 leader_election.cc:302] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 221ecd53825f4eaa9a2b5dc60db2f137; no voters:
I20260807 06:33:16.174023 4029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:16.174059 4029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:16.174158 4033 leader_election.cc:288] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260807 06:33:16.174798 4029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42265 } }
I20260807 06:33:16.174891 4029 leader_election.cc:302] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 221ecd53825f4eaa9a2b5dc60db2f137; no voters:
I20260807 06:33:16.174932 4029 leader_election.cc:288] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:16.174949 4033 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 06:33:16.175077 4033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:16.175125 4033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [term 1 LEADER]: Becoming Leader. State: Replica: 221ecd53825f4eaa9a2b5dc60db2f137, State: Running, Role: LEADER
I20260807 06:33:16.175197 4029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:16.175201 4033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42265 } }
I20260807 06:33:16.175561 4041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42265 } } }
I20260807 06:33:16.175575 4038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 221ecd53825f4eaa9a2b5dc60db2f137. Latest consensus state: current_term: 1 leader_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221ecd53825f4eaa9a2b5dc60db2f137" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42265 } } }
I20260807 06:33:16.175750 4038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:16.175720 4041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:16.176625 4046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:16.177419 4046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:16.179033 4046 catalog_manager.cc:1347] Generated new cluster ID: ca2d136af88942f49d1301e947306777
I20260807 06:33:16.179088 4046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:16.184619 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.174137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.2.243.62:42265)
0807 06:33:16.174317 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.174328 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.174424 (+ 96us) server_negotiation.cc:407] Connection header received
0807 06:33:16.174574 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.174579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.174621 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.174727 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:16.175601 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.177171 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.177963 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.178603 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.178725 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.181268 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:16.181272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:16.181273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:16.181284 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:16.182898 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.183286 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.183289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.183289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.183391 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.183598 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.183599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.183600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.183818 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:16.183979 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.184079 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.184338 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":61,"threads_started":1}
I20260807 06:33:16.184665 4046 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:16.185314 4046 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:16.190933 4046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 221ecd53825f4eaa9a2b5dc60db2f137: Generated new TSK 0
I20260807 06:33:16.191098 4046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:16.219971 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42265
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:16.329269 4071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:16.329564 4071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:16.329625 4071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:16.330796 4071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:16.330863 4071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:16.330911 4071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:16.332563 4071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42265
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:16.332815 4071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:16.333043 4071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:16.338215 4080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:16.338227 4078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:16.338472 4071 server_base.cc:1034] running on GCE node
W20260807 06:33:16.338261 4084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:16.338897 4071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:16.339207 4071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:16.340412 4071 hybrid_clock.cc:648] HybridClock initialized: now 1786084396340391 us; error 46 us; skew 500 ppm
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:16.343500 4071 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:16.343984 4071 webserver.cc:460] Webserver started at http://127.2.243.1:42839/ using document root <none> and password file <none>
I20260807 06:33:16.344223 4071 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:16.344274 4071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:16.344400 4071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:16.345553 4071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "45badeeb40964c4581f1c5ada17b2754"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "fa88d73babdc0d92e867c8310b753d1b"
server_key_iv: "58f0d163c80d9eec103e27e77e49131a"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.345922 4071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "45badeeb40964c4581f1c5ada17b2754"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "fa88d73babdc0d92e867c8310b753d1b"
server_key_iv: "58f0d163c80d9eec103e27e77e49131a"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.347318 4071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:16.348134 4097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.348307 4071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:16.348409 4071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "45badeeb40964c4581f1c5ada17b2754"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "fa88d73babdc0d92e867c8310b753d1b"
server_key_iv: "58f0d163c80d9eec103e27e77e49131a"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.348476 4071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:16.367914 4071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:16.370136 4071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:16.370303 4071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:16.370556 4071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:16.370877 4071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:16.370930 4071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.370972 4071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:16.370996 4071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.377108 4071 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40657
I20260807 06:33:16.377152 4267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40657 every 8 connection(s)
I20260807 06:33:16.377501 4071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:16.386059 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4071
I20260807 06:33:16.386157 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:16.386363 3020 external_mini_cluster.cc:1428] Setting key d0a2fd1181f627b8c24de21b215f1731
I20260807 06:33:16.388301 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.378307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:38031 (local address 127.2.243.62:42265)
0807 06:33:16.378370 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.378372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.378604 (+ 232us) server_negotiation.cc:407] Connection header received
0807 06:33:16.379681 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.379682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.379699 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.379752 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:16.380394 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.380969 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.381970 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.382189 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.382272 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.384671 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:16.384673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:16.384673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:16.384681 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:16.386508 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.387004 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.387006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.387006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.387085 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.387403 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.387404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.387405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.387605 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:16.387669 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.388032 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.388137 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:33:16.388499 4270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.378280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42265 (local address 127.2.243.1:38031)
0807 06:33:16.378470 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.378515 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.379559 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.379827 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.379830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.380092 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:16.380296 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.380301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.381064 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.381065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.381864 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.381866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.382285 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.382286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.382420 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.382998 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:16.383007 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:16.384550 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:16.386613 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:16.386634 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:16.386636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:16.386905 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:16.387162 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:16.387163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:16.387163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:16.387320 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:16.387680 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:16.387687 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:16.387866 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.387986 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.388205 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260807 06:33:16.389371 4268 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42265
I20260807 06:33:16.389492 4268 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:16.389680 4268 heartbeater.cc:511] Master 127.2.243.62:42265 requested a full tablet report, sending...
I20260807 06:33:16.390231 3978 ts_manager.cc:194] Registered new tserver with Master: 45badeeb40964c4581f1c5ada17b2754 (127.2.243.1:40657)
I20260807 06:33:16.390857 3978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:38031
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:16.415681 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42265
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:16.498373 4274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:16.498615 4274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:16.498656 4274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:16.499827 4274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:16.499891 4274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:16.499935 4274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:16.501466 4274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42265
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:16.501741 4274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:16.501969 4274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:16.507304 4280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:16.507309 4281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:16.507329 4283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:16.507346 4274 server_base.cc:1034] running on GCE node
I20260807 06:33:16.507679 4274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:16.507920 4274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:16.509099 4274 hybrid_clock.cc:648] HybridClock initialized: now 1786084396509082 us; error 46 us; skew 500 ppm
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:16.511608 4274 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:16.512065 4274 webserver.cc:460] Webserver started at http://127.2.243.2:43557/ using document root <none> and password file <none>
I20260807 06:33:16.512311 4274 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:16.512377 4274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:16.512575 4274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:16.513597 4274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "2b7a5ab6bf87465990a5b14bf6593333"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "5615b8b9ebe7b88e1d3f9fb6bef98e92"
server_key_iv: "de75caefd7729f0d3eee035a6ac9f5b7"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.513943 4274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "2b7a5ab6bf87465990a5b14bf6593333"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "5615b8b9ebe7b88e1d3f9fb6bef98e92"
server_key_iv: "de75caefd7729f0d3eee035a6ac9f5b7"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.515286 4274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:16.516008 4295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.516208 4274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:16.516306 4274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "2b7a5ab6bf87465990a5b14bf6593333"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "5615b8b9ebe7b88e1d3f9fb6bef98e92"
server_key_iv: "de75caefd7729f0d3eee035a6ac9f5b7"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.516381 4274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:16.530380 4274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:16.532829 4274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:16.532948 4274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:16.533124 4274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:16.533373 4274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:16.533402 4274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.533425 4274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:16.533437 4274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.539364 4274 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:45305
I20260807 06:33:16.539428 4476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:45305 every 8 connection(s)
I20260807 06:33:16.539762 4274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:16.541013 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4274
I20260807 06:33:16.541087 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:16.541257 3020 external_mini_cluster.cc:1428] Setting key 7c3f9293c1cd92a43715b59c94d3a4b8
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
I20260807 06:33:16.550689 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.540539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:46347 (local address 127.2.243.62:42265)
0807 06:33:16.540588 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.540590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.540815 (+ 225us) server_negotiation.cc:407] Connection header received
0807 06:33:16.542045 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.542046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.542058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.542115 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:16.542740 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.543263 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.544846 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.545077 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.545179 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.547729 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:16.547731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:16.547731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:16.547740 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:16.549242 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.549671 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.549672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.549672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.549733 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.549990 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.549991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.549991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.550113 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:16.550185 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.550483 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.550595 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:16.550876 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.540530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42265 (local address 127.2.243.2:46347)
0807 06:33:16.540699 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.540734 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.541906 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.542186 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.542190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.542454 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:16.542651 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.542655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.543355 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.543356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.544738 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.544742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.545200 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.545201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.545357 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.545891 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:16.545903 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:16.547624 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:16.549342 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:16.549350 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:16.549353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:16.549592 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:16.549806 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:16.549807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:16.549807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:16.549919 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:16.550189 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:16.550196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:16.550352 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.550462 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.550640 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260807 06:33:16.551393 4477 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42265
I20260807 06:33:16.551499 4477 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:16.551684 4477 heartbeater.cc:511] Master 127.2.243.62:42265 requested a full tablet report, sending...
I20260807 06:33:16.552073 3978 ts_manager.cc:194] Registered new tserver with Master: 2b7a5ab6bf87465990a5b14bf6593333 (127.2.243.2:45305)
I20260807 06:33:16.552619 3978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:46347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:16.571235 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42265
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:16.654834 4483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:16.655076 4483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:16.655103 4483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:16.656288 4483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:16.656334 4483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:16.656363 4483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:16.657861 4483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42265
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:16.658069 4483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:16.658250 4483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:16.663381 4491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:16.663357 4492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:16.663357 4494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:16.663610 4483 server_base.cc:1034] running on GCE node
I20260807 06:33:16.663906 4483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:16.664155 4483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:16.665336 4483 hybrid_clock.cc:648] HybridClock initialized: now 1786084396665328 us; error 37 us; skew 500 ppm
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:16.668017 4483 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:16.668567 4483 webserver.cc:460] Webserver started at http://127.2.243.3:44071/ using document root <none> and password file <none>
I20260807 06:33:16.668812 4483 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:16.668876 4483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:16.669016 4483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:16.670094 4483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "aaf7e7d1d6634c47bd992c803603648d"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "758db2508311bfc0119aa6073cfb0836"
server_key_iv: "5f5530a18780f1daa5747529ed17f892"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.670461 4483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "aaf7e7d1d6634c47bd992c803603648d"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "758db2508311bfc0119aa6073cfb0836"
server_key_iv: "5f5530a18780f1daa5747529ed17f892"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.671806 4483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:16.672628 4509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.672849 4483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:16.672953 4483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "aaf7e7d1d6634c47bd992c803603648d"
format_stamp: "Formatted at 2026-08-07 06:33:16 on dist-test-slave-rwrg"
server_key: "758db2508311bfc0119aa6073cfb0836"
server_key_iv: "5f5530a18780f1daa5747529ed17f892"
server_key_version: "encryptionkey@0"
I20260807 06:33:16.673038 4483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:16.695421 4483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:16.697399 4483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:16.697558 4483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:16.697781 4483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:16.698138 4483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:16.698201 4483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.698243 4483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:16.698271 4483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:16.704586 4483 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:35259
I20260807 06:33:16.704640 4699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:35259 every 8 connection(s)
I20260807 06:33:16.704993 4483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:16.706912 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4483
I20260807 06:33:16.707017 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:16.707204 3020 external_mini_cluster.cc:1428] Setting key 5fa7987aa93b95ea3bb08c2d16d1221c
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:16.714471 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.705864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:49823 (local address 127.2.243.62:42265)
0807 06:33:16.705912 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.705914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.705983 (+ 69us) server_negotiation.cc:407] Connection header received
0807 06:33:16.706954 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.706955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.706969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.707064 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:16.707932 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.708574 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.709692 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.709867 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.709927 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.711931 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:16.711933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:16.711933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:16.711939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:16.713310 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.713680 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.713681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.713681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.713731 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.713931 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.713931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.713932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.714040 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:16.714090 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.714315 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.714388 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:16.714702 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.705708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42265 (local address 127.2.243.3:49823)
0807 06:33:16.705861 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.705905 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.706834 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.707169 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.707172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.707536 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:16.707843 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.707850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.708668 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.708668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.709623 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.709626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.709941 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.709941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.710036 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.710504 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:16.710512 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:16.711848 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:16.713381 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:16.713387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:16.713389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:16.713616 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:16.713785 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:16.713785 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:16.713785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:16.713881 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:16.714091 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:16.714094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:16.714207 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.714290 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.714442 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260807 06:33:16.715195 4700 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42265
I20260807 06:33:16.715292 4700 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:16.715467 4700 heartbeater.cc:511] Master 127.2.243.62:42265 requested a full tablet report, sending...
I20260807 06:33:16.715950 3978 ts_manager.cc:194] Registered new tserver with Master: aaf7e7d1d6634c47bd992c803603648d (127.2.243.3:35259)
I20260807 06:33:16.716312 3978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:49823
I20260807 06:33:16.719192 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:33:16.727245 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.721269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.2.243.62:42265)
0807 06:33:16.721338 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.721340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.721428 (+ 88us) server_negotiation.cc:407] Connection header received
0807 06:33:16.721484 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.721485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.721494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.721547 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:16.722126 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.722597 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.723330 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.723508 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.723571 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.724873 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:16.724874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:16.724874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:16.724881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:16.726148 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.726481 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.726482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.726482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.726532 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.726698 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.726699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.726699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.726834 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:16.726908 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.727058 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.727120 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:33:16.730165 3978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:16.731983 3978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:16.738909 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.735258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38616 (local address 127.2.243.1:40657)
0807 06:33:16.735452 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.735457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.735582 (+ 125us) server_negotiation.cc:407] Connection header received
0807 06:33:16.735625 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.735627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.735641 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.735710 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:16.735913 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.736960 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.737970 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.738503 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.738608 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.738672 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.738793 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.738831 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
I20260807 06:33:16.738900 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.735005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40657 (local address 127.0.0.1:38616)
0807 06:33:16.735463 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.735473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.735525 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.735788 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.735789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.735791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:16.735837 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.735839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.737050 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.737052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.737878 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.737881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.738610 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.738611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.738690 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.738704 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.738772 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.738826 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":103,"threads_started":1}
I20260807 06:33:16.739217 4720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.734825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:45305 (local address 127.0.0.1:55992)
0807 06:33:16.735244 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.735254 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.735307 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.735532 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.735536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.735539 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:16.735611 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.735613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.736959 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.736960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.738051 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.738053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.738996 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.738997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.739062 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.739066 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.739097 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.739138 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260807 06:33:16.739311 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.734808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:35259 (local address 127.0.0.1:47782)
0807 06:33:16.735114 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.735160 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.735237 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.735558 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.735559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.735561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:16.735611 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.735613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.736733 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.736734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.738050 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.738053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.739047 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.739048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.739106 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.739109 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.739172 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.739205 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":37,"thread_start_us":55,"threads_started":1}
I20260807 06:33:16.739346 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.734914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.2.243.3:35259)
0807 06:33:16.735116 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.735121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.735328 (+ 207us) server_negotiation.cc:407] Connection header received
0807 06:33:16.735374 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.735377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.735395 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.735468 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:16.735701 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.736636 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.738161 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.738891 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.738964 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.739019 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.739203 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.739248 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260807 06:33:16.739600 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.735057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.2.243.2:45305)
0807 06:33:16.735234 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.735239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.735318 (+ 79us) server_negotiation.cc:407] Connection header received
0807 06:33:16.735371 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.735373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.735389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.735516 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:16.735677 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.736861 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.738157 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.738892 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.739402 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.739467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.739497 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.739528 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260807 06:33:16.740053 4594 tablet_service.cc:1467] Processing CreateTablet for tablet 4ab1d8e2cf9a451782033e01bda3dd81 (DEFAULT_TABLE table=test-table [id=1afeebef48064556bf0de19ecf416474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:16.740146 4175 tablet_service.cc:1467] Processing CreateTablet for tablet 4ab1d8e2cf9a451782033e01bda3dd81 (DEFAULT_TABLE table=test-table [id=1afeebef48064556bf0de19ecf416474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:16.740451 4594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ab1d8e2cf9a451782033e01bda3dd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:16.740572 4175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ab1d8e2cf9a451782033e01bda3dd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:16.741076 4373 tablet_service.cc:1467] Processing CreateTablet for tablet 4ab1d8e2cf9a451782033e01bda3dd81 (DEFAULT_TABLE table=test-table [id=1afeebef48064556bf0de19ecf416474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:16.741394 4373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ab1d8e2cf9a451782033e01bda3dd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:16.743503 4727 tablet_bootstrap.cc:492] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d: Bootstrap starting.
I20260807 06:33:16.743971 4728 tablet_bootstrap.cc:492] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754: Bootstrap starting.
I20260807 06:33:16.744196 4727 tablet_bootstrap.cc:654] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:16.744769 4729 tablet_bootstrap.cc:492] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333: Bootstrap starting.
I20260807 06:33:16.744836 4728 tablet_bootstrap.cc:654] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:16.745039 4727 log.cc:824] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:16.745405 4728 log.cc:824] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:16.745488 4729 tablet_bootstrap.cc:654] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:16.745898 4727 tablet_bootstrap.cc:492] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d: No bootstrap required, opened a new log
I20260807 06:33:16.745977 4727 ts_tablet_manager.cc:1397] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 06:33:16.746064 4729 log.cc:824] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:16.746160 4728 tablet_bootstrap.cc:492] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754: No bootstrap required, opened a new log
I20260807 06:33:16.746230 4728 ts_tablet_manager.cc:1397] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:16.746770 4729 tablet_bootstrap.cc:492] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333: No bootstrap required, opened a new log
I20260807 06:33:16.746842 4729 ts_tablet_manager.cc:1397] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:16.747861 4728 raft_consensus.cc:348] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.748028 4728 raft_consensus.cc:374] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:16.748071 4728 raft_consensus.cc:729] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45badeeb40964c4581f1c5ada17b2754, State: Initialized, Role: FOLLOWER
I20260807 06:33:16.748190 4728 consensus_queue.cc:260] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.748102 4727 raft_consensus.cc:348] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.748286 4727 raft_consensus.cc:374] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:16.748329 4727 raft_consensus.cc:729] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaf7e7d1d6634c47bd992c803603648d, State: Initialized, Role: FOLLOWER
I20260807 06:33:16.748456 4727 consensus_queue.cc:260] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.748479 4728 ts_tablet_manager.cc:1428] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:33:16.748703 4268 heartbeater.cc:503] Master 127.2.243.62:42265 was elected leader, sending a full tablet report...
I20260807 06:33:16.748754 4727 ts_tablet_manager.cc:1428] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 06:33:16.748879 4700 heartbeater.cc:503] Master 127.2.243.62:42265 was elected leader, sending a full tablet report...
I20260807 06:33:16.748883 4729 raft_consensus.cc:348] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.749037 4729 raft_consensus.cc:374] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:16.749069 4729 raft_consensus.cc:729] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7a5ab6bf87465990a5b14bf6593333, State: Initialized, Role: FOLLOWER
I20260807 06:33:16.749176 4729 consensus_queue.cc:260] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.749486 4729 ts_tablet_manager.cc:1428] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:33:16.749502 4477 heartbeater.cc:503] Master 127.2.243.62:42265 was elected leader, sending a full tablet report...
W20260807 06:33:16.790349 4479 tablet.cc:2369] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:33:16.878294 4269 tablet.cc:2369] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:16.932250 4743 raft_consensus.cc:482] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:16.932425 4743 raft_consensus.cc:504] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.933063 4743 leader_election.cc:288] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b7a5ab6bf87465990a5b14bf6593333 (127.2.243.2:45305), 45badeeb40964c4581f1c5ada17b2754 (127.2.243.1:40657)
I20260807 06:33:16.935899 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.933349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:32787 (local address 127.2.243.2:45305)
0807 06:33:16.933398 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.933401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.933414 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:33:16.933495 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.933496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.933510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.933581 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:16.933741 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.934219 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.935198 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.935562 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.935640 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.935652 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.935791 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.935829 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:33:16.935899 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.933290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:45305 (local address 127.2.243.3:32787)
0807 06:33:16.933354 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.933367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.933415 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.933587 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.933588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.933590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:16.933665 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.933667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.934302 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.934303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.935114 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.935116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.935654 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.935655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.935725 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.935728 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.935787 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.935833 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 06:33:16.936174 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.933420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:56335 (local address 127.2.243.1:40657)
0807 06:33:16.933464 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.933466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.933585 (+ 119us) server_negotiation.cc:407] Connection header received
0807 06:33:16.933680 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.933681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.933697 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.933766 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:16.933922 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.934392 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.935399 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.935741 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.935843 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.935853 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.936060 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.936101 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:33:16.936388 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.933290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40657 (local address 127.2.243.3:56335)
0807 06:33:16.933496 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:16.933511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:16.933604 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:16.933769 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:16.933770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:16.933772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:16.933856 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.933857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.934480 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.934481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.935317 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:16.935319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.935852 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.935853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:16.935949 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.935953 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:16.936252 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 06:33:16.936304 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":14,"thread_start_us":116,"threads_started":1}
I20260807 06:33:16.936362 4400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab1d8e2cf9a451782033e01bda3dd81" candidate_uuid: "aaf7e7d1d6634c47bd992c803603648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7a5ab6bf87465990a5b14bf6593333" is_pre_election: true
I20260807 06:33:16.936672 4400 raft_consensus.cc:2454] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaf7e7d1d6634c47bd992c803603648d in term 0.
I20260807 06:33:16.936898 4202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab1d8e2cf9a451782033e01bda3dd81" candidate_uuid: "aaf7e7d1d6634c47bd992c803603648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45badeeb40964c4581f1c5ada17b2754" is_pre_election: true
I20260807 06:33:16.937021 4515 leader_election.cc:302] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7a5ab6bf87465990a5b14bf6593333, aaf7e7d1d6634c47bd992c803603648d; no voters:
I20260807 06:33:16.937088 4202 raft_consensus.cc:2454] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaf7e7d1d6634c47bd992c803603648d in term 0.
I20260807 06:33:16.937184 4743 raft_consensus.cc:2781] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:16.937265 4743 raft_consensus.cc:482] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:16.937296 4743 raft_consensus.cc:3037] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:16.938423 4743 raft_consensus.cc:504] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.938589 4743 leader_election.cc:288] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [CANDIDATE]: Term 1 election: Requested vote from peers 2b7a5ab6bf87465990a5b14bf6593333 (127.2.243.2:45305), 45badeeb40964c4581f1c5ada17b2754 (127.2.243.1:40657)
I20260807 06:33:16.938791 4202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab1d8e2cf9a451782033e01bda3dd81" candidate_uuid: "aaf7e7d1d6634c47bd992c803603648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45badeeb40964c4581f1c5ada17b2754"
I20260807 06:33:16.938791 4400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ab1d8e2cf9a451782033e01bda3dd81" candidate_uuid: "aaf7e7d1d6634c47bd992c803603648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7a5ab6bf87465990a5b14bf6593333"
I20260807 06:33:16.938874 4202 raft_consensus.cc:3037] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:16.938874 4400 raft_consensus.cc:3037] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:16.939764 4400 raft_consensus.cc:2454] T 4ab1d8e2cf9a451782033e01bda3dd81 P 2b7a5ab6bf87465990a5b14bf6593333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaf7e7d1d6634c47bd992c803603648d in term 1.
I20260807 06:33:16.939764 4202 raft_consensus.cc:2454] T 4ab1d8e2cf9a451782033e01bda3dd81 P 45badeeb40964c4581f1c5ada17b2754 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaf7e7d1d6634c47bd992c803603648d in term 1.
I20260807 06:33:16.939934 4510 leader_election.cc:302] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45badeeb40964c4581f1c5ada17b2754, aaf7e7d1d6634c47bd992c803603648d; no voters:
I20260807 06:33:16.940076 4743 raft_consensus.cc:2781] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:16.940240 4743 raft_consensus.cc:686] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [term 1 LEADER]: Becoming Leader. State: Replica: aaf7e7d1d6634c47bd992c803603648d, State: Running, Role: LEADER
I20260807 06:33:16.940342 4743 consensus_queue.cc:237] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } }
I20260807 06:33:16.944922 3978 catalog_manager.cc:5495] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d reported cstate change: term changed from 0 to 1, leader changed from <none> to aaf7e7d1d6634c47bd992c803603648d (127.2.243.3). New cstate: current_term: 1 leader_uuid: "aaf7e7d1d6634c47bd992c803603648d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf7e7d1d6634c47bd992c803603648d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b7a5ab6bf87465990a5b14bf6593333" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 45305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45badeeb40964c4581f1c5ada17b2754" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40657 } health_report { overall_health: UNKNOWN } } }
W20260807 06:33:16.955588 4701 tablet.cc:2369] T 4ab1d8e2cf9a451782033e01bda3dd81 P aaf7e7d1d6634c47bd992c803603648d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:16.968379 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.961130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.2.243.1:40657)
0807 06:33:16.961204 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.961206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.961267 (+ 61us) server_negotiation.cc:407] Connection header received
0807 06:33:16.961316 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.961317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.961326 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.961379 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:16.961945 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.962413 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.963105 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.963278 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.963327 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.965662 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:16.965671 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:16.965671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:16.965677 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:16.967328 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.967663 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.967666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.967666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.967729 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.967887 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.967887 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.967888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.968023 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:16.968137 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.968251 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.968313 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:16.992031 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:16.985254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.2.243.1:40657)
0807 06:33:16.985334 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:16.985337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:16.985405 (+ 68us) server_negotiation.cc:407] Connection header received
0807 06:33:16.985450 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:16.985451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:16.985459 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:16.985510 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:16.986072 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.986543 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.987312 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:16.987507 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:16.987574 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:16.989846 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:16.989848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:16.989848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:16.989855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:16.991071 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.991380 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.991381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.991381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.991427 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:16.991579 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:16.991580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:16.991580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:16.991661 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:16.991728 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:16.991854 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 06:33:16.991913 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 06:33:16 dist-test-slave-rwrg krb5kdc[3905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:33:17 dist-test-slave-rwrg krb5kdc[3905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:17.015162 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:17.008200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.2.243.1:40657)
0807 06:33:17.008273 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:17.008275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:17.008327 (+ 52us) server_negotiation.cc:407] Connection header received
0807 06:33:17.008380 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:17.008381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:17.008390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:17.008439 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:17.009000 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:17.009478 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:17.010238 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:17.010433 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:17.010524 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:17.012865 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:17.012866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:17.012866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:17.012883 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:17.014284 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:17.014561 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:17.014562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:17.014562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:17.014613 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:17.014762 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:17.014763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:17.014763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:17.014858 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:17.014931 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:17.015013 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 06:33:17.015069 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 06:33:17.015915 4139 tablet_service.cc:2908] Rejecting scan request for tablet 4ab1d8e2cf9a451782033e01bda3dd81: Uninitialized: clean time has not yet been initialized
I20260807 06:33:17.017205 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4071
I20260807 06:33:17.029408 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4274
I20260807 06:33:17.040985 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4483
I20260807 06:33:17.050848 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3924
2026-08-07T06:33:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2389 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:17 dist-test-slave-rwrg krb5kdc[4767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:17 dist-test-slave-rwrg krb5kdc[4767](info): set up 2 sockets
Aug 07 06:33:17 dist-test-slave-rwrg krb5kdc[4767](info): commencing operation
krb5kdc: starting...
W20260807 06:33:18.303916 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:18 dist-test-slave-rwrg krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:18.426978 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:40497
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:40497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:18.509455 4786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:18.509677 4786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:18.509699 4786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:18.510871 4786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:18.510912 4786 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:33:18.510932 4786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:18.510964 4786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:18.510993 4786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:18.511010 4786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:18.513082 4786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:40497
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:40497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:18.513363 4786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:18.513619 4786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:18.518918 4794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:18.518932 4796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:18.519040 4786 server_base.cc:1034] running on GCE node
W20260807 06:33:18.518939 4791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:18.519367 4786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:18.519706 4786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:18.520874 4786 hybrid_clock.cc:648] HybridClock initialized: now 1786084398520854 us; error 35 us; skew 500 ppm
Aug 07 06:33:18 dist-test-slave-rwrg krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:18.523470 4786 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:18.523890 4786 webserver.cc:460] Webserver started at http://127.2.243.62:44995/ using document root <none> and password file <none>
I20260807 06:33:18.524132 4786 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:18.524197 4786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:18.524343 4786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:18.525440 4786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "e9825438b68f4a2f89fc5ac17616ee24"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "e277c4f8f434b3873ebd923b5228d2ed"
server_key_iv: "a3a569bf740261fff0114af9cf26f650"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.525808 4786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "e9825438b68f4a2f89fc5ac17616ee24"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "e277c4f8f434b3873ebd923b5228d2ed"
server_key_iv: "a3a569bf740261fff0114af9cf26f650"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.527141 4786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:18.527913 4806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:18.528084 4786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:18.528184 4786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "e9825438b68f4a2f89fc5ac17616ee24"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "e277c4f8f434b3873ebd923b5228d2ed"
server_key_iv: "a3a569bf740261fff0114af9cf26f650"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.528262 4786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:18.547199 4786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:18.549516 4786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:18.549693 4786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:18.554694 4786 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:40497
I20260807 06:33:18.554742 4907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:40497 every 8 connection(s)
I20260807 06:33:18.555087 4786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:18.555754 4909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:18.558182 4909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24: Bootstrap starting.
I20260807 06:33:18.558780 4909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:18.559079 4909 log.cc:824] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:18.559795 4909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24: No bootstrap required, opened a new log
I20260807 06:33:18.562068 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4786
I20260807 06:33:18.562178 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:18.562104 4909 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9825438b68f4a2f89fc5ac17616ee24" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40497 } }
I20260807 06:33:18.562285 4909 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:18.562351 4909 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9825438b68f4a2f89fc5ac17616ee24, State: Initialized, Role: FOLLOWER
I20260807 06:33:18.562404 3020 external_mini_cluster.cc:1428] Setting key c85deed2de1e99ad1497b8117802f8c7
I20260807 06:33:18.562517 4909 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9825438b68f4a2f89fc5ac17616ee24" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40497 } }
I20260807 06:33:18.562597 4909 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:18.562629 4909 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:18.562680 4909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:18.563424 4909 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9825438b68f4a2f89fc5ac17616ee24" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40497 } }
I20260807 06:33:18.563544 4909 leader_election.cc:302] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9825438b68f4a2f89fc5ac17616ee24; no voters:
I20260807 06:33:18.563697 4909 leader_election.cc:288] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:18.563747 4921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:18.563906 4921 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [term 1 LEADER]: Becoming Leader. State: Replica: e9825438b68f4a2f89fc5ac17616ee24, State: Running, Role: LEADER
I20260807 06:33:18.563946 4909 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:18.564015 4921 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9825438b68f4a2f89fc5ac17616ee24" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40497 } }
I20260807 06:33:18.564410 4921 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9825438b68f4a2f89fc5ac17616ee24. Latest consensus state: current_term: 1 leader_uuid: "e9825438b68f4a2f89fc5ac17616ee24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9825438b68f4a2f89fc5ac17616ee24" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40497 } } }
I20260807 06:33:18.564536 4921 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:18.564419 4922 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9825438b68f4a2f89fc5ac17616ee24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9825438b68f4a2f89fc5ac17616ee24" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40497 } } }
I20260807 06:33:18.564935 4932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:18.565407 4932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:18.564934 4922 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:18.567358 4932 catalog_manager.cc:1347] Generated new cluster ID: d9afa11a835f454ba53e1a9c90f82463
I20260807 06:33:18.567413 4932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:18 dist-test-slave-rwrg krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:18.574460 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.562918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.2.243.62:40497)
0807 06:33:18.563192 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:18.563202 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:18.563238 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:33:18.563353 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:18.563360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:18.563419 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:18.563584 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:18.564590 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.566205 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.567191 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.567705 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.567849 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.570255 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:18.570259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:18.570259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:18.570278 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:18.572811 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.573233 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.573236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.573236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.573338 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.573558 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.573559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.573559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.573793 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:18.573974 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:18.574048 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 06:33:18.574296 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":130,"threads_started":1}
I20260807 06:33:18.591552 4932 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:18.592269 4932 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:18.599906 4932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24: Generated new TSK 0
I20260807 06:33:18.600098 4932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:18.629715 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:40497
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:18.714054 4949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:18.714258 4949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:18.714284 4949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:18.715548 4949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:18.715597 4949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:18.715619 4949 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:18.715637 4949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:18.717142 4949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:40497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:18.717406 4949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:18.717636 4949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:18.722905 4961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:18.722968 4956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:18.723050 4958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:18.723094 4949 server_base.cc:1034] running on GCE node
I20260807 06:33:18.723351 4949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:18.723621 4949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:18.724790 4949 hybrid_clock.cc:648] HybridClock initialized: now 1786084398724771 us; error 39 us; skew 500 ppm
Aug 07 06:33:18 dist-test-slave-rwrg krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:18.727404 4949 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:18.727841 4949 webserver.cc:460] Webserver started at http://127.2.243.1:41071/ using document root <none> and password file <none>
I20260807 06:33:18.728073 4949 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:18.728120 4949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:18.728264 4949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:18.729399 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "4777388a97bf4ba9b4584c71d95e73d9"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "d1f21a9ff86200b30e19fdc38323743a"
server_key_iv: "d8880af242cdd443f3c33610a4cef9aa"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.729758 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "4777388a97bf4ba9b4584c71d95e73d9"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "d1f21a9ff86200b30e19fdc38323743a"
server_key_iv: "d8880af242cdd443f3c33610a4cef9aa"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.731173 4949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:18.731921 4969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:18.732164 4949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:18.732271 4949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "4777388a97bf4ba9b4584c71d95e73d9"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "d1f21a9ff86200b30e19fdc38323743a"
server_key_iv: "d8880af242cdd443f3c33610a4cef9aa"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.732347 4949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:18.742246 4949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:18.744117 4949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:18.744270 4949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:18.744933 4949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:18.744997 4949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:18.745054 4949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:18.745088 4949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:18.752156 4949 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:46615
I20260807 06:33:18.752233 5161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:46615 every 8 connection(s)
Aug 07 06:33:18 dist-test-slave-rwrg krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:18.752625 4949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:18.755249 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4949
I20260807 06:33:18.755345 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:18.755556 3020 external_mini_cluster.cc:1428] Setting key fbd830b5d2482a992433d7e9a9095e10
I20260807 06:33:18.756098 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.745619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:45671 (local address 127.2.243.62:40497)
0807 06:33:18.745665 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:18.745667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:18.745876 (+ 209us) server_negotiation.cc:407] Connection header received
0807 06:33:18.746432 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:18.746433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:18.746451 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:18.746518 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:18.747419 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.748270 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.749907 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.750255 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.750345 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.753142 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:18.753143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:18.753143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:18.753152 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:18.754564 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.754931 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.754932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.754933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.754985 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.755283 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.755283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.755284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.755433 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:18.755489 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:18.755899 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 06:33:18.755999 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:18.756213 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.745600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:40497 (local address 127.2.243.1:45671)
0807 06:33:18.745765 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:18.745800 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:18.746299 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:18.746616 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:18.746621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:18.747005 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:18.747309 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.747315 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.748393 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.748395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.749804 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.749807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.750350 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.750350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.750455 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.751033 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:18.751057 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:18.753025 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:18.754639 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.754645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.754647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.754865 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.755039 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.755039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.755039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.755213 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.755501 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:18.755508 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:18.755739 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 06:33:18.755860 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 06:33:18.756015 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260807 06:33:18.756724 5162 heartbeater.cc:347] Connected to a master server at 127.2.243.62:40497
I20260807 06:33:18.756821 5162 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:18.757020 5162 heartbeater.cc:511] Master 127.2.243.62:40497 requested a full tablet report, sending...
I20260807 06:33:18.757586 4831 ts_manager.cc:194] Registered new tserver with Master: 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615)
I20260807 06:33:18.758373 4831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:45671
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
I20260807 06:33:18.766984 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.757842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:53421 (local address 127.2.243.62:40497)
0807 06:33:18.757881 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:18.757883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:18.760700 (+ 2817us) server_negotiation.cc:407] Connection header received
0807 06:33:18.760749 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:18.760750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:18.760761 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:18.760823 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:18.761213 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.761712 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.762810 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.763006 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.763067 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.764258 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:18.764259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:18.764260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:18.764268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:18.765926 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.766250 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.766251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.766251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.766309 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.766468 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.766468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.766468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.766610 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:18.766662 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:18.766816 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 06:33:18.766912 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:33:18.767005 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.757797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:40497 (local address 127.2.243.1:53421)
0807 06:33:18.760615 (+ 2818us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:18.760628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:18.760695 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:18.760831 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:18.760832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:18.761067 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:18.761137 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.761138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.761824 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.761825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.762722 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.762724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.763088 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.763089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.763171 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.763749 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:18.763755 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:18.764156 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:18.766017 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.766018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.766019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.766187 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.766364 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.766365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.766365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.766417 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.766687 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:18.766688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:18.766731 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 06:33:18.766795 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:33:18.766907 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2787,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:18.783285 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:40497
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:18.865350 5179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:18.865602 5179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:18.865628 5179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:18.866813 5179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:18.866859 5179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:18.866880 5179 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:18.866899 5179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:18.868362 5179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:40497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:18.868673 5179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:18.868912 5179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:18.873966 5186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:18.873910 5187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:18.873975 5189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:18.874166 5179 server_base.cc:1034] running on GCE node
I20260807 06:33:18.874373 5179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:18.874576 5179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:18.875712 5179 hybrid_clock.cc:648] HybridClock initialized: now 1786084398875701 us; error 28 us; skew 500 ppm
Aug 07 06:33:18 dist-test-slave-rwrg krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:18.878264 5179 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:18.878741 5179 webserver.cc:460] Webserver started at http://127.2.243.2:39511/ using document root <none> and password file <none>
I20260807 06:33:18.878990 5179 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:18.879040 5179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:18.879176 5179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:18.880224 5179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "3c81caff9655422caec120acb4ed1283"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "a2574a9755696c483eb714d786119867"
server_key_iv: "22623c4a967155812345951927f787d4"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.880615 5179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "3c81caff9655422caec120acb4ed1283"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "a2574a9755696c483eb714d786119867"
server_key_iv: "22623c4a967155812345951927f787d4"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.881924 5179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:18.882645 5198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:18.882813 5179 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:18.882916 5179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "3c81caff9655422caec120acb4ed1283"
format_stamp: "Formatted at 2026-08-07 06:33:18 on dist-test-slave-rwrg"
server_key: "a2574a9755696c483eb714d786119867"
server_key_iv: "22623c4a967155812345951927f787d4"
server_key_version: "encryptionkey@0"
I20260807 06:33:18.882985 5179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:18.896128 5179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:18.898303 5179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:18.898459 5179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:18.898975 5179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:18.899044 5179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:18.899088 5179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:18.899115 5179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:18.906425 5179 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:41287
Aug 07 06:33:18 dist-test-slave-rwrg krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:18.906549 5386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:41287 every 8 connection(s)
I20260807 06:33:18.906862 5179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:18.908389 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5179
I20260807 06:33:18.908493 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:18.908699 3020 external_mini_cluster.cc:1428] Setting key 887d60bd7f434662149d3efdac3bb24d
I20260807 06:33:18.910220 5216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.899484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:40497 (local address 127.2.243.2:39457)
0807 06:33:18.899703 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:18.899737 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:18.900224 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:18.900552 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:18.900557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:18.900949 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:18.901259 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.901266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.902287 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.902288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.903793 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.903798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.904309 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.904310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.904477 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.905005 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:18.905017 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:18.906956 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:18.908723 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.908729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.908731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.909045 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.909248 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.909248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.909249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.909404 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.909716 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:18.909721 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:18.909834 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 06:33:18.909921 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:33:18.910062 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":106,"threads_started":1}
I20260807 06:33:18.910234 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.899500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:39457 (local address 127.2.243.62:40497)
0807 06:33:18.899591 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:18.899594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:18.899819 (+ 225us) server_negotiation.cc:407] Connection header received
0807 06:33:18.900334 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:18.900335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:18.900348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:18.900416 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:18.901369 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.902191 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.903892 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.904211 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.904291 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.907057 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:18.907059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:18.907059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:18.907070 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:18.908629 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.909115 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.909116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.909117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.909173 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.909489 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.909490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.909490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.909644 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:18.909723 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:18.909969 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 06:33:18.910087 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 06:33:18.910782 5387 heartbeater.cc:347] Connected to a master server at 127.2.243.62:40497
I20260807 06:33:18.910883 5387 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:18.911077 5387 heartbeater.cc:511] Master 127.2.243.62:40497 requested a full tablet report, sending...
I20260807 06:33:18.911444 4832 ts_manager.cc:194] Registered new tserver with Master: 3c81caff9655422caec120acb4ed1283 (127.2.243.2:41287)
I20260807 06:33:18.911967 4832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:39457
I20260807 06:33:18.918501 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.912429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:43715 (local address 127.2.243.62:40497)
0807 06:33:18.912549 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:18.912551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:18.912624 (+ 73us) server_negotiation.cc:407] Connection header received
0807 06:33:18.912687 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:18.912688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:18.912697 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:18.912750 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:18.913118 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.913564 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.914648 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.914833 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.914886 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.915923 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:18.915924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:18.915924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:18.915930 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:18.917374 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.917771 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.917772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.917772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.917821 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:18.918020 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:18.918020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:18.918021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:18.918156 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:18.918208 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:18.918388 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 06:33:18.918442 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 06:33:18.918635 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:18.912370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:40497 (local address 127.2.243.2:43715)
0807 06:33:18.912560 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:18.912572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:18.912638 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:18.912755 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:18.912755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:18.912970 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:18.913042 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.913043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.913664 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.913665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.914568 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:18.914572 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:18.914910 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:18.914912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:18.914988 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:18.915404 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:18.915410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:18.915837 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:18.917454 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.917456 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.917456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.917696 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.917884 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:18.917885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:18.917886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:18.917960 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:18.918226 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:18.918227 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:18.918281 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 06:33:18.918375 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:33:18.918503 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:18.939352 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:40497
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:19.023823 5402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:19.024029 5402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:19.024053 5402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:19.025305 5402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:19.025349 5402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:19.025370 5402 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:19.025388 5402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:19.026818 5402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:40497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:19.027022 5402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:19.027195 5402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:19.032389 5411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:19.032395 5412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:19.032404 5414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:19.032615 5402 server_base.cc:1034] running on GCE node
I20260807 06:33:19.032859 5402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:19.033164 5402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:19.034328 5402 hybrid_clock.cc:648] HybridClock initialized: now 1786084399034320 us; error 35 us; skew 500 ppm
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084399, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:19.037178 5402 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:19.037673 5402 webserver.cc:460] Webserver started at http://127.2.243.3:41671/ using document root <none> and password file <none>
I20260807 06:33:19.037885 5402 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:19.037931 5402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:19.038043 5402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:19.039101 5402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "47714090c2bf4f3485cbe4f9c7073c2f"
format_stamp: "Formatted at 2026-08-07 06:33:19 on dist-test-slave-rwrg"
server_key: "c06dcc0407a4b878398945a037466799"
server_key_iv: "b8e33d061d82455c7ea448c57c6d78f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:19.039428 5402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "47714090c2bf4f3485cbe4f9c7073c2f"
format_stamp: "Formatted at 2026-08-07 06:33:19 on dist-test-slave-rwrg"
server_key: "c06dcc0407a4b878398945a037466799"
server_key_iv: "b8e33d061d82455c7ea448c57c6d78f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:19.040906 5402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:19.041767 5424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:19.042011 5402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:19.042094 5402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "47714090c2bf4f3485cbe4f9c7073c2f"
format_stamp: "Formatted at 2026-08-07 06:33:19 on dist-test-slave-rwrg"
server_key: "c06dcc0407a4b878398945a037466799"
server_key_iv: "b8e33d061d82455c7ea448c57c6d78f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:19.042181 5402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:19.069235 5402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:19.071632 5402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:19.071784 5402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:19.072367 5402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:19.072434 5402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:19.072479 5402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:19.072610 5402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084399, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:19.080564 5402 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:44361
I20260807 06:33:19.080662 5612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:44361 every 8 connection(s)
I20260807 06:33:19.080991 5402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:19.083554 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.072790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:37011 (local address 127.2.243.62:40497)
0807 06:33:19.072845 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.072848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.073147 (+ 299us) server_negotiation.cc:407] Connection header received
0807 06:33:19.073854 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.073855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.073867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.073961 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:19.074976 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.075489 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.077223 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.077434 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.077520 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.080157 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:19.080159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:19.080159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:19.080168 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:19.081985 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.082483 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.082485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.082485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.082561 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.082830 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.082831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.082832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.082993 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:19.083084 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.083360 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.083457 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:33:19.083739 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.072812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:40497 (local address 127.2.243.3:37011)
0807 06:33:19.072976 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.073034 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.073744 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.074057 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.074062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.074515 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:19.074830 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.074838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.075573 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.075574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.077132 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.077135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.077554 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.077555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.077676 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.078247 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:19.078258 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:19.080035 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:19.082091 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:19.082099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:19.082101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:19.082380 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:19.082631 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:19.082632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:19.082632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:19.082744 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:19.083072 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:19.083080 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:19.083218 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.083333 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.083508 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260807 06:33:19.084254 5613 heartbeater.cc:347] Connected to a master server at 127.2.243.62:40497
I20260807 06:33:19.084344 5613 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:19.084578 5613 heartbeater.cc:511] Master 127.2.243.62:40497 requested a full tablet report, sending...
I20260807 06:33:19.085021 4831 ts_manager.cc:194] Registered new tserver with Master: 47714090c2bf4f3485cbe4f9c7073c2f (127.2.243.3:44361)
I20260807 06:33:19.085143 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5402
I20260807 06:33:19.085216 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:19.085372 3020 external_mini_cluster.cc:1428] Setting key ea47e62e2d8e925213a36f8a1d6c4db3
I20260807 06:33:19.085523 4831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:37011
I20260807 06:33:19.086362 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:33:19.092722 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.085761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:51397 (local address 127.2.243.62:40497)
0807 06:33:19.085798 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.085800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.086519 (+ 719us) server_negotiation.cc:407] Connection header received
0807 06:33:19.086620 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.086621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.086633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.086709 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:19.087071 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.087550 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.088409 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.088681 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.088738 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.089805 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:19.089807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:19.089807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:19.089814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:19.091531 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.091872 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.091873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.091873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.091937 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.092137 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.092138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.092139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.092280 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:19.092350 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.092599 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.092664 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 06:33:19.092839 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.085769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:40497 (local address 127.2.243.3:51397)
0807 06:33:19.086431 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.086460 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.086547 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.086755 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.086755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.086926 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:19.086988 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.086989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.087649 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.087650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.088304 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.088306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.088760 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.088761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.088847 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.089286 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:19.089291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:19.089710 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:19.091620 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:19.091621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:19.091621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:19.091794 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:19.092011 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:19.092012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:19.092012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:19.092072 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:19.092385 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:19.092386 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:19.092440 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.092574 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.092695 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":72,"threads_started":1}
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[4767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:19.111663 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.104614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.2.243.62:40497)
0807 06:33:19.104675 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.104677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.104688 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:19.104718 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.104719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.104729 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.104784 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:19.105333 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.105812 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.106559 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.106744 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.106833 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.109209 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:19.109210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:19.109210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:19.109226 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:19.110436 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.110827 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.110829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.110829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.110931 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.111124 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.111125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.111125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.111235 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:19.111338 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.111471 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.111560 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:33:19.113453 4831 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:19.115036 4831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:19.121812 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.117715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41186 (local address 127.2.243.1:46615)
0807 06:33:19.118217 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.118222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.118234 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:19.118277 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.118279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.118295 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.118358 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:19.118610 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.119301 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.120684 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.121367 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.121476 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.121544 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.121667 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.121716 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":95,"threads_started":1}
I20260807 06:33:19.121801 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.117619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:46615 (local address 127.0.0.1:41186)
0807 06:33:19.117988 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.118031 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.118100 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.118483 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.118483 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.118485 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:19.118534 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.118536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.119416 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.119417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.120582 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.120585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.121471 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.121472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.121554 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.121569 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.121652 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.121716 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":85,"threads_started":1}
I20260807 06:33:19.122217 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.117843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:41287 (local address 127.0.0.1:58648)
0807 06:33:19.117993 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.118032 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.118102 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.118430 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.118431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.118434 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:19.118524 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.118542 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.119867 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.119869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.120998 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.121002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.121947 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.121948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.122019 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.122024 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.122094 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.122135 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":4,"thread_start_us":88,"threads_started":1}
I20260807 06:33:19.122465 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.117787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.2.243.3:44361)
0807 06:33:19.117931 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.117936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.118112 (+ 176us) server_negotiation.cc:407] Connection header received
0807 06:33:19.118235 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.118237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.118255 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.118311 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:19.118582 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.119725 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.120957 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.121694 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.121768 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.121820 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.122332 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.122373 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260807 06:33:19.122673 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.117622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44361 (local address 127.0.0.1:38400)
0807 06:33:19.117988 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.118031 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.118140 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.118420 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.118423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.118425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:19.118490 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.118492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.119833 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.119834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.120870 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.120872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.122150 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.122152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.122223 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.122228 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.122562 (+ 334us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.122608 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":50,"threads_started":1}
I20260807 06:33:19.122663 5065 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1f3397ac0348a9a47bc539c6b3b02c (DEFAULT_TABLE table=test-table [id=8fdba5a5aef64d4abf5ff43a71f53425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:19.123010 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1f3397ac0348a9a47bc539c6b3b02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.124850 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.117975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.2.243.2:41287)
0807 06:33:19.118246 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.118250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.118258 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:19.118295 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.118297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.118312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.118638 (+ 326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:19.118706 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.119754 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.121095 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.121836 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.121939 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.122000 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.122106 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.122148 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":91,"threads_started":1}
I20260807 06:33:19.125232 5506 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1f3397ac0348a9a47bc539c6b3b02c (DEFAULT_TABLE table=test-table [id=8fdba5a5aef64d4abf5ff43a71f53425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:19.125552 5282 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1f3397ac0348a9a47bc539c6b3b02c (DEFAULT_TABLE table=test-table [id=8fdba5a5aef64d4abf5ff43a71f53425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:19.125684 5506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1f3397ac0348a9a47bc539c6b3b02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.125913 5282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1f3397ac0348a9a47bc539c6b3b02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.126416 5645 tablet_bootstrap.cc:492] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Bootstrap starting.
I20260807 06:33:19.127136 5645 tablet_bootstrap.cc:654] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.127514 5645 log.cc:824] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:19.128357 5645 tablet_bootstrap.cc:492] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: No bootstrap required, opened a new log
I20260807 06:33:19.128433 5645 ts_tablet_manager.cc:1397] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:19.129105 5647 tablet_bootstrap.cc:492] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Bootstrap starting.
I20260807 06:33:19.129678 5648 tablet_bootstrap.cc:492] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Bootstrap starting.
I20260807 06:33:19.129843 5647 tablet_bootstrap.cc:654] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.130229 5647 log.cc:824] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:19.130556 5645 raft_consensus.cc:348] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.130731 5648 tablet_bootstrap.cc:654] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.130755 5645 raft_consensus.cc:374] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.130818 5645 raft_consensus.cc:729] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4777388a97bf4ba9b4584c71d95e73d9, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.130947 5645 consensus_queue.cc:260] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.131091 5648 log.cc:824] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:19.131253 5647 tablet_bootstrap.cc:492] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: No bootstrap required, opened a new log
I20260807 06:33:19.131268 5645 ts_tablet_manager.cc:1428] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:33:19.131392 5162 heartbeater.cc:503] Master 127.2.243.62:40497 was elected leader, sending a full tablet report...
I20260807 06:33:19.131390 5647 ts_tablet_manager.cc:1397] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:19.131944 5648 tablet_bootstrap.cc:492] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: No bootstrap required, opened a new log
I20260807 06:33:19.132023 5648 ts_tablet_manager.cc:1397] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:19.133538 5647 raft_consensus.cc:348] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.133701 5647 raft_consensus.cc:374] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.133739 5647 raft_consensus.cc:729] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c81caff9655422caec120acb4ed1283, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.133846 5647 consensus_queue.cc:260] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.133967 5648 raft_consensus.cc:348] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.134123 5647 ts_tablet_manager.cc:1428] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:33:19.134133 5648 raft_consensus.cc:374] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.134169 5648 raft_consensus.cc:729] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47714090c2bf4f3485cbe4f9c7073c2f, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.134164 5387 heartbeater.cc:503] Master 127.2.243.62:40497 was elected leader, sending a full tablet report...
I20260807 06:33:19.134272 5648 consensus_queue.cc:260] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.134523 5648 ts_tablet_manager.cc:1428] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:19.134579 5613 heartbeater.cc:503] Master 127.2.243.62:40497 was elected leader, sending a full tablet report...
I20260807 06:33:19.154171 5656 raft_consensus.cc:482] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:19.154315 5656 raft_consensus.cc:504] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.154695 5656 leader_election.cc:288] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615), 47714090c2bf4f3485cbe4f9c7073c2f (127.2.243.3:44361)
W20260807 06:33:19.157430 5388 tablet.cc:2369] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:19.158263 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.154945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:56161 (local address 127.2.243.1:46615)
0807 06:33:19.154989 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.154991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.155000 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:19.155033 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.155034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.155045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.155107 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:19.155312 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.155899 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.157460 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.157887 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.157971 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.157983 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.158146 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.158190 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:33:19.158275 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.154971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:37301 (local address 127.2.243.3:44361)
0807 06:33:19.155032 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.155034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.155202 (+ 168us) server_negotiation.cc:407] Connection header received
0807 06:33:19.155230 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.155231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.155244 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.155298 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:19.155502 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.156044 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.157418 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.157916 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.157989 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.158000 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.158169 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.158208 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:33:19.158294 5216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.154859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:46615 (local address 127.2.243.2:56161)
0807 06:33:19.154916 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.154930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.154974 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.155122 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.155123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.155124 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:19.155213 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.155216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.155989 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.155990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.157302 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.157303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.157971 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.157972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.158070 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.158073 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.158137 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.158186 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":4}
I20260807 06:33:19.158314 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.154903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44361 (local address 127.2.243.2:37301)
0807 06:33:19.155097 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.155106 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.155158 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.155310 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.155311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.155313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:19.155390 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.155392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.156148 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.156149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.157296 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.157299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.157994 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.157995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.158097 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.158100 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.158162 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.158208 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":27,"thread_start_us":111,"threads_started":1}
I20260807 06:33:19.158651 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1f3397ac0348a9a47bc539c6b3b02c" candidate_uuid: "3c81caff9655422caec120acb4ed1283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4777388a97bf4ba9b4584c71d95e73d9" is_pre_election: true
I20260807 06:33:19.158689 5537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1f3397ac0348a9a47bc539c6b3b02c" candidate_uuid: "3c81caff9655422caec120acb4ed1283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" is_pre_election: true
I20260807 06:33:19.158818 5537 raft_consensus.cc:2454] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c81caff9655422caec120acb4ed1283 in term 0.
I20260807 06:33:19.158818 5093 raft_consensus.cc:2454] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c81caff9655422caec120acb4ed1283 in term 0.
I20260807 06:33:19.159080 5206 leader_election.cc:302] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c81caff9655422caec120acb4ed1283, 47714090c2bf4f3485cbe4f9c7073c2f; no voters:
I20260807 06:33:19.159222 5656 raft_consensus.cc:2781] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:19.159279 5656 raft_consensus.cc:482] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:19.159302 5656 raft_consensus.cc:3037] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.160386 5656 raft_consensus.cc:504] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.160554 5656 leader_election.cc:288] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [CANDIDATE]: Term 1 election: Requested vote from peers 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615), 47714090c2bf4f3485cbe4f9c7073c2f (127.2.243.3:44361)
I20260807 06:33:19.160705 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1f3397ac0348a9a47bc539c6b3b02c" candidate_uuid: "3c81caff9655422caec120acb4ed1283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4777388a97bf4ba9b4584c71d95e73d9"
I20260807 06:33:19.160718 5537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1f3397ac0348a9a47bc539c6b3b02c" candidate_uuid: "3c81caff9655422caec120acb4ed1283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47714090c2bf4f3485cbe4f9c7073c2f"
I20260807 06:33:19.160804 5093 raft_consensus.cc:3037] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.160800 5537 raft_consensus.cc:3037] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.161545 5093 raft_consensus.cc:2454] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c81caff9655422caec120acb4ed1283 in term 1.
I20260807 06:33:19.161660 5537 raft_consensus.cc:2454] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c81caff9655422caec120acb4ed1283 in term 1.
I20260807 06:33:19.161757 5205 leader_election.cc:302] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c81caff9655422caec120acb4ed1283, 4777388a97bf4ba9b4584c71d95e73d9; no voters:
I20260807 06:33:19.161845 5656 raft_consensus.cc:2781] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:19.162000 5656 raft_consensus.cc:686] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 1 LEADER]: Becoming Leader. State: Replica: 3c81caff9655422caec120acb4ed1283, State: Running, Role: LEADER
I20260807 06:33:19.162101 5656 consensus_queue.cc:237] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.162992 4831 catalog_manager.cc:5495] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c81caff9655422caec120acb4ed1283 (127.2.243.2). New cstate: current_term: 1 leader_uuid: "3c81caff9655422caec120acb4ed1283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260807 06:33:19.172154 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.169935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.2.243.2:41287)
0807 06:33:19.169988 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.169990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.169997 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:33:19.170072 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.170072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.170088 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.170149 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:19.170275 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.170731 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.171455 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.171646 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.171695 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.171877 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:19.171964 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:19.172031 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.172067 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.172083 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:33:19.173908 5093 raft_consensus.cc:1261] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Refusing update from remote peer 3c81caff9655422caec120acb4ed1283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:19.174250 5661 consensus_queue.cc:1035] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:19.175846 5537 raft_consensus.cc:1261] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Refusing update from remote peer 3c81caff9655422caec120acb4ed1283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:19.176091 5656 consensus_queue.cc:1035] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:19.176754 5668 mvcc.cc:205] Tried to move back new op lower bound from 7315801699013468160 to 7315801698968608768. Current Snapshot: MvccSnapshot[applied={T|T < 7315801699013468160}]
I20260807 06:33:19.177368 5673 mvcc.cc:205] Tried to move back new op lower bound from 7315801699013468160 to 7315801698968608768. Current Snapshot: MvccSnapshot[applied={T|T < 7315801699013468160}]
I20260807 06:33:19.181100 4831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:19.181206 4831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:19.181988 4831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24: Sending DeleteTablet for 3 replicas of tablet 5c1f3397ac0348a9a47bc539c6b3b02c
I20260807 06:33:19.182457 5506 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1f3397ac0348a9a47bc539c6b3b02c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:19 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:38400
I20260807 06:33:19.183578 5065 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1f3397ac0348a9a47bc539c6b3b02c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:19 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:41186
I20260807 06:33:19.183607 5282 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1f3397ac0348a9a47bc539c6b3b02c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:19 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:58648
I20260807 06:33:19.183833 5684 tablet_replica.cc:330] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: stopping tablet replica
I20260807 06:33:19.183935 5684 raft_consensus.cc:2229] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:19.184064 5682 tablet_replica.cc:330] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: stopping tablet replica
I20260807 06:33:19.184068 5684 raft_consensus.cc:2258] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:19.184216 5683 tablet_replica.cc:330] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: stopping tablet replica
I20260807 06:33:19.184245 4831 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 06:33:19.184320 5683 raft_consensus.cc:2229] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 06:33:19.184393 4831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:19.184482 5683 raft_consensus.cc:2258] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:19.184662 5682 raft_consensus.cc:2229] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:19.185289 5683 ts_tablet_manager.cc:1905] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:19.185711 5684 ts_tablet_manager.cc:1905] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:19.185930 5282 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13d068be4a45caab62b7f4f7e26210 (DEFAULT_TABLE table=test-table [id=461df4eaac7d464984090f5d25163c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:19.186057 5282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13d068be4a45caab62b7f4f7e26210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.187376 5647 tablet_bootstrap.cc:492] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: Bootstrap starting.
I20260807 06:33:19.187830 5647 tablet_bootstrap.cc:654] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.185786 5506 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13d068be4a45caab62b7f4f7e26210 (DEFAULT_TABLE table=test-table [id=461df4eaac7d464984090f5d25163c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:19.188046 5506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13d068be4a45caab62b7f4f7e26210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.188161 5682 raft_consensus.cc:2258] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:19.188259 5065 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13d068be4a45caab62b7f4f7e26210 (DEFAULT_TABLE table=test-table [id=461df4eaac7d464984090f5d25163c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:19.188366 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13d068be4a45caab62b7f4f7e26210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.189129 5682 ts_tablet_manager.cc:1905] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:19.189607 5645 tablet_bootstrap.cc:492] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Bootstrap starting.
I20260807 06:33:19.189986 5648 tablet_bootstrap.cc:492] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Bootstrap starting.
I20260807 06:33:19.187837 5684 ts_tablet_manager.cc:1918] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:19.190055 5645 tablet_bootstrap.cc:654] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.190078 5684 log.cc:1196] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/5c1f3397ac0348a9a47bc539c6b3b02c
I20260807 06:33:19.190407 5684 ts_tablet_manager.cc:1939] T 5c1f3397ac0348a9a47bc539c6b3b02c P 3c81caff9655422caec120acb4ed1283: Deleting consensus metadata
I20260807 06:33:19.190444 5648 tablet_bootstrap.cc:654] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.190788 4809 catalog_manager.cc:4841] TS 3c81caff9655422caec120acb4ed1283 (127.2.243.2:41287): tablet 5c1f3397ac0348a9a47bc539c6b3b02c (table test-table [id=8fdba5a5aef64d4abf5ff43a71f53425]) successfully deleted
I20260807 06:33:19.191128 5683 ts_tablet_manager.cc:1918] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:19.191205 5683 log.cc:1196] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/5c1f3397ac0348a9a47bc539c6b3b02c
I20260807 06:33:19.191337 5682 ts_tablet_manager.cc:1918] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:19.191423 5682 log.cc:1196] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/5c1f3397ac0348a9a47bc539c6b3b02c
I20260807 06:33:19.191463 5683 ts_tablet_manager.cc:1939] T 5c1f3397ac0348a9a47bc539c6b3b02c P 4777388a97bf4ba9b4584c71d95e73d9: Deleting consensus metadata
I20260807 06:33:19.191679 5682 ts_tablet_manager.cc:1939] T 5c1f3397ac0348a9a47bc539c6b3b02c P 47714090c2bf4f3485cbe4f9c7073c2f: Deleting consensus metadata
I20260807 06:33:19.191874 4809 catalog_manager.cc:4841] TS 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615): tablet 5c1f3397ac0348a9a47bc539c6b3b02c (table test-table [id=8fdba5a5aef64d4abf5ff43a71f53425]) successfully deleted
I20260807 06:33:19.192065 4810 catalog_manager.cc:4841] TS 47714090c2bf4f3485cbe4f9c7073c2f (127.2.243.3:44361): tablet 5c1f3397ac0348a9a47bc539c6b3b02c (table test-table [id=8fdba5a5aef64d4abf5ff43a71f53425]) successfully deleted
I20260807 06:33:19.192097 5645 tablet_bootstrap.cc:492] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: No bootstrap required, opened a new log
I20260807 06:33:19.192138 5645 ts_tablet_manager.cc:1397] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 06:33:19.192284 5645 raft_consensus.cc:348] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.192353 5645 raft_consensus.cc:374] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.192373 5645 raft_consensus.cc:729] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4777388a97bf4ba9b4584c71d95e73d9, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.192417 5645 consensus_queue.cc:260] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.192543 5645 ts_tablet_manager.cc:1428] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:19.192956 5647 tablet_bootstrap.cc:492] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: No bootstrap required, opened a new log
I20260807 06:33:19.193004 5647 ts_tablet_manager.cc:1397] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 06:33:19.193153 5647 raft_consensus.cc:348] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.193225 5647 raft_consensus.cc:374] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.193245 5647 raft_consensus.cc:729] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c81caff9655422caec120acb4ed1283, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.193300 5647 consensus_queue.cc:260] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.193382 5647 ts_tablet_manager.cc:1428] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:19.196903 5648 tablet_bootstrap.cc:492] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: No bootstrap required, opened a new log
I20260807 06:33:19.196969 5648 ts_tablet_manager.cc:1397] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 06:33:19.197113 5648 raft_consensus.cc:348] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.197207 5648 raft_consensus.cc:374] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.197233 5648 raft_consensus.cc:729] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47714090c2bf4f3485cbe4f9c7073c2f, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.197278 5648 consensus_queue.cc:260] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.197378 5648 ts_tablet_manager.cc:1428] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:19.249400 5658 raft_consensus.cc:482] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:19.249517 5658 raft_consensus.cc:504] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.249835 5658 leader_election.cc:288] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615), 3c81caff9655422caec120acb4ed1283 (127.2.243.2:41287)
I20260807 06:33:19.253377 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.250045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:46615 (local address 127.2.243.3:49969)
0807 06:33:19.250112 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.250152 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.250215 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.250426 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.250427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.250429 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:19.250518 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.250520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.251383 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.251384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.252523 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.252524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.253089 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.253090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.253209 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.253213 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.253265 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.253305 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":5}
I20260807 06:33:19.253400 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.250143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:49969 (local address 127.2.243.1:46615)
0807 06:33:19.250189 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.250191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.250201 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:19.250300 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.250301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.250312 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.250362 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:19.250586 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.251293 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.252645 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.253001 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.253089 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.253100 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.253285 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.253333 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 06:33:19.253407 5167 tablet.cc:2369] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:19.253686 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13d068be4a45caab62b7f4f7e26210" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4777388a97bf4ba9b4584c71d95e73d9" is_pre_election: true
I20260807 06:33:19.253782 5093 raft_consensus.cc:2454] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 0.
I20260807 06:33:19.253831 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.250167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:41287 (local address 127.2.243.3:51861)
0807 06:33:19.250310 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.250326 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.250375 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.250515 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.250515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.250516 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:19.250589 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.250590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.251221 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.251223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.252497 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.252500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.253254 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.253255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.253587 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.253592 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.253679 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.253720 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260807 06:33:19.253892 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.250332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:51861 (local address 127.2.243.2:41287)
0807 06:33:19.250381 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.250384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.250396 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:19.250427 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.250427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.250436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.250503 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:19.250682 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.251140 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.252596 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.253130 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.253220 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.253231 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.253791 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.253830 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:33:19.253959 5427 leader_election.cc:302] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47714090c2bf4f3485cbe4f9c7073c2f, 4777388a97bf4ba9b4584c71d95e73d9; no voters:
I20260807 06:33:19.254079 5658 raft_consensus.cc:2781] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:19.254120 5658 raft_consensus.cc:482] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:19.254149 5658 raft_consensus.cc:3037] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.254261 5317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13d068be4a45caab62b7f4f7e26210" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c81caff9655422caec120acb4ed1283" is_pre_election: true
I20260807 06:33:19.254383 5317 raft_consensus.cc:2454] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 0.
I20260807 06:33:19.255026 5658 raft_consensus.cc:504] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.255154 5658 leader_election.cc:288] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 election: Requested vote from peers 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615), 3c81caff9655422caec120acb4ed1283 (127.2.243.2:41287)
I20260807 06:33:19.255339 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13d068be4a45caab62b7f4f7e26210" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4777388a97bf4ba9b4584c71d95e73d9"
I20260807 06:33:19.255363 5317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13d068be4a45caab62b7f4f7e26210" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c81caff9655422caec120acb4ed1283"
I20260807 06:33:19.255415 5093 raft_consensus.cc:3037] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.255424 5317 raft_consensus.cc:3037] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.256161 5093 raft_consensus.cc:2454] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 1.
I20260807 06:33:19.256170 5317 raft_consensus.cc:2454] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 1.
I20260807 06:33:19.256407 5427 leader_election.cc:302] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c81caff9655422caec120acb4ed1283, 47714090c2bf4f3485cbe4f9c7073c2f; no voters:
I20260807 06:33:19.256533 5658 raft_consensus.cc:2781] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:19.256707 5658 raft_consensus.cc:686] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 LEADER]: Becoming Leader. State: Replica: 47714090c2bf4f3485cbe4f9c7073c2f, State: Running, Role: LEADER
I20260807 06:33:19.256786 5658 consensus_queue.cc:237] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.257557 4831 catalog_manager.cc:5495] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 47714090c2bf4f3485cbe4f9c7073c2f (127.2.243.3). New cstate: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } health_report { overall_health: HEALTHY } } }
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084398, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:19.284387 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.277260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44622 (local address 127.2.243.62:40497)
0807 06:33:19.277323 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.277325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.277438 (+ 113us) server_negotiation.cc:407] Connection header received
0807 06:33:19.277517 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.277518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.277532 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.277590 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:19.278044 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.278528 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.279212 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.279396 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.279464 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.281753 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:19.281755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:19.281755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:19.281760 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:19.283182 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.283516 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.283517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.283518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.283608 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.283825 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.283826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.283826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.283929 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:19.283995 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.284187 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.284300 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":32}
I20260807 06:33:19.284382 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.277213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:40497 (local address 127.0.0.1:44622)
0807 06:33:19.277360 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.277370 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.277457 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.277595 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.277596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.277913 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:19.277964 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.277966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.278603 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.278604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.279116 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.279117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.279462 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.279463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.279513 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.280047 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:19.280052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:19.281654 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:19.283259 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:19.283267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:19.283269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:19.283446 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:19.283694 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:19.283694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:19.283695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:19.283754 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:19.283999 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:19.284007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:19.284048 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.284116 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.284242 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260807 06:33:19.286772 4831 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:44622:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:33:19.287097 4831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:19.288681 5506 tablet_service.cc:1467] Processing CreateTablet for tablet a2543cc69ce74426b2b4fa088ae1ef3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8553188c3cf84960be01fc0bcab5b85d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:19.288722 5065 tablet_service.cc:1467] Processing CreateTablet for tablet a2543cc69ce74426b2b4fa088ae1ef3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8553188c3cf84960be01fc0bcab5b85d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:19.288905 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2543cc69ce74426b2b4fa088ae1ef3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.288903 5506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2543cc69ce74426b2b4fa088ae1ef3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.290113 5648 tablet_bootstrap.cc:492] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: Bootstrap starting.
I20260807 06:33:19.290213 5282 tablet_service.cc:1467] Processing CreateTablet for tablet a2543cc69ce74426b2b4fa088ae1ef3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8553188c3cf84960be01fc0bcab5b85d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:19.290352 5282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2543cc69ce74426b2b4fa088ae1ef3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:19.290438 5645 tablet_bootstrap.cc:492] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: Bootstrap starting.
I20260807 06:33:19.290549 5648 tablet_bootstrap.cc:654] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.290832 5645 tablet_bootstrap.cc:654] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.291509 5645 tablet_bootstrap.cc:492] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: No bootstrap required, opened a new log
I20260807 06:33:19.291567 5645 ts_tablet_manager.cc:1397] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:19.291719 5645 raft_consensus.cc:348] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.291795 5645 raft_consensus.cc:374] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.291817 5645 raft_consensus.cc:729] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4777388a97bf4ba9b4584c71d95e73d9, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.291862 5645 consensus_queue.cc:260] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.291975 5645 ts_tablet_manager.cc:1428] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:19.292078 5647 tablet_bootstrap.cc:492] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: Bootstrap starting.
I20260807 06:33:19.292328 5648 tablet_bootstrap.cc:492] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: No bootstrap required, opened a new log
I20260807 06:33:19.292393 5648 ts_tablet_manager.cc:1397] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:19.292558 5647 tablet_bootstrap.cc:654] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:19.292588 5648 raft_consensus.cc:348] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.292670 5648 raft_consensus.cc:374] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.292690 5648 raft_consensus.cc:729] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47714090c2bf4f3485cbe4f9c7073c2f, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.292732 5648 consensus_queue.cc:260] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.292725 5651 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.292805 5651 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:19.292816 5648 ts_tablet_manager.cc:1428] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:19.292919 5658 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.292986 5658 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:19.293450 5647 tablet_bootstrap.cc:492] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: No bootstrap required, opened a new log
I20260807 06:33:19.293512 5647 ts_tablet_manager.cc:1397] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:19.293659 5647 raft_consensus.cc:348] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.293732 5647 raft_consensus.cc:374] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:19.293753 5647 raft_consensus.cc:729] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c81caff9655422caec120acb4ed1283, State: Initialized, Role: FOLLOWER
I20260807 06:33:19.293798 5647 consensus_queue.cc:260] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.293891 5647 ts_tablet_manager.cc:1428] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:19.294032 5656 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.294102 5656 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:19.331199 5658 raft_consensus.cc:482] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:19.331337 5658 raft_consensus.cc:504] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.331593 5658 leader_election.cc:288] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615), 3c81caff9655422caec120acb4ed1283 (127.2.243.2:41287)
W20260807 06:33:19.331655 5614 tablet.cc:2369] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:19.331813 5317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2543cc69ce74426b2b4fa088ae1ef3c" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c81caff9655422caec120acb4ed1283" is_pre_election: true
I20260807 06:33:19.331820 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2543cc69ce74426b2b4fa088ae1ef3c" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4777388a97bf4ba9b4584c71d95e73d9" is_pre_election: true
I20260807 06:33:19.331943 5093 raft_consensus.cc:2454] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 0.
I20260807 06:33:19.331947 5317 raft_consensus.cc:2454] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 0.
I20260807 06:33:19.332145 5427 leader_election.cc:302] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47714090c2bf4f3485cbe4f9c7073c2f, 4777388a97bf4ba9b4584c71d95e73d9; no voters:
I20260807 06:33:19.332257 5658 raft_consensus.cc:2781] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:19.332304 5658 raft_consensus.cc:482] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:19.332332 5658 raft_consensus.cc:3037] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.333563 5658 raft_consensus.cc:504] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.333715 5658 leader_election.cc:288] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 election: Requested vote from peers 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615), 3c81caff9655422caec120acb4ed1283 (127.2.243.2:41287)
I20260807 06:33:19.333923 5093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2543cc69ce74426b2b4fa088ae1ef3c" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4777388a97bf4ba9b4584c71d95e73d9"
I20260807 06:33:19.333966 5317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2543cc69ce74426b2b4fa088ae1ef3c" candidate_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c81caff9655422caec120acb4ed1283"
I20260807 06:33:19.334030 5317 raft_consensus.cc:3037] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.334014 5093 raft_consensus.cc:3037] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:19.334771 5093 raft_consensus.cc:2454] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 1.
I20260807 06:33:19.334913 5317 raft_consensus.cc:2454] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47714090c2bf4f3485cbe4f9c7073c2f in term 1.
I20260807 06:33:19.334966 5427 leader_election.cc:302] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47714090c2bf4f3485cbe4f9c7073c2f, 4777388a97bf4ba9b4584c71d95e73d9; no voters:
I20260807 06:33:19.335043 5658 raft_consensus.cc:2781] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:19.335095 5658 raft_consensus.cc:686] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 LEADER]: Becoming Leader. State: Replica: 47714090c2bf4f3485cbe4f9c7073c2f, State: Running, Role: LEADER
I20260807 06:33:19.335145 5658 consensus_queue.cc:237] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } }
I20260807 06:33:19.335472 5693 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: TxnStatusTablet state changed. Reason: New leader 47714090c2bf4f3485cbe4f9c7073c2f. Latest consensus state: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.335569 5693 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:19.335784 5711 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:33:19.335954 5711 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:33:19.335875 4831 catalog_manager.cc:5495] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 47714090c2bf4f3485cbe4f9c7073c2f (127.2.243.3). New cstate: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } health_report { overall_health: HEALTHY } } }
I20260807 06:33:19.342907 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.340228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.2.243.3:44361)
0807 06:33:19.340285 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.340287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.340296 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:19.340375 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.340375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.340390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.340450 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:19.340616 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.341074 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.342133 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.342324 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.342377 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.342606 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:19.342693 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:19.342746 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.342838 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.342855 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:33:19.342968 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.340130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44361 (local address 127.0.0.1:38402)
0807 06:33:19.340221 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.340234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.340281 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.340456 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.340457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.340458 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:19.340551 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.340554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.341162 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.341163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.342044 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.342048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.342414 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.342415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.342491 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.342511 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:19.342763 (+ 252us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:19.342771 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.342838 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.342890 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 06:33:19.344163 5093 raft_consensus.cc:1261] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Refusing update from remote peer 47714090c2bf4f3485cbe4f9c7073c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:19.344285 5317 raft_consensus.cc:1261] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Refusing update from remote peer 47714090c2bf4f3485cbe4f9c7073c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:19.344421 5658 consensus_queue.cc:1035] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:19.344640 5658 consensus_queue.cc:1035] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:19.345022 5651 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: TxnStatusTablet state changed. Reason: New leader 47714090c2bf4f3485cbe4f9c7073c2f. Latest consensus state: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.345137 5651 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:19.345348 5656 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: TxnStatusTablet state changed. Reason: New leader 47714090c2bf4f3485cbe4f9c7073c2f. Latest consensus state: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.345445 5656 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:19.346001 5678 mvcc.cc:205] Tried to move back new op lower bound from 7315801699711770624 to 7315801699677216768. Current Snapshot: MvccSnapshot[applied={T|T < 7315801699711770624}]
I20260807 06:33:19.346268 5658 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.346343 5658 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:19.346313 5651 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.346398 5651 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 4777388a97bf4ba9b4584c71d95e73d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:19.346652 5693 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.346733 5693 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 47714090c2bf4f3485cbe4f9c7073c2f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:19.347584 5656 tablet_replica.cc:439] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 } } peers { permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 } } peers { permanent_uuid: "47714090c2bf4f3485cbe4f9c7073c2f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44361 } } }
I20260807 06:33:19.347687 5656 tablet_replica.cc:442] T a2543cc69ce74426b2b4fa088ae1ef3c P 3c81caff9655422caec120acb4ed1283: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:19.351421 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.348932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.2.243.3:44361)
0807 06:33:19.348968 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.348970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.349039 (+ 69us) server_negotiation.cc:407] Connection header received
0807 06:33:19.349111 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.349112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.349134 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.349178 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:19.349329 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.349768 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.350800 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.350982 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.351041 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.351214 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:19.351239 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:19.351272 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.351341 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.351356 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:33:19.356196 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.353969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:43981 (local address 127.2.243.3:44361)
0807 06:33:19.354005 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.354007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.354074 (+ 67us) server_negotiation.cc:407] Connection header received
0807 06:33:19.354132 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.354133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.354142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.354195 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:19.354311 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.354766 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.355512 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.355695 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.355763 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.355926 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:19.355954 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:19.356013 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.356072 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.356093 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":15}
I20260807 06:33:19.356209 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.353915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44361 (local address 127.2.243.3:43981)
0807 06:33:19.354028 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:19.354038 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:19.354088 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:19.354197 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:19.354197 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:19.354198 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:19.354252 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.354253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.354850 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.354851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.355435 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:19.355436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.355762 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.355763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:19.355824 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.355842 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:19.356014 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:19.356016 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:19.356067 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:33:19.356112 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":3}
I20260807 06:33:19.358451 5093 raft_consensus.cc:1261] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Refusing update from remote peer 47714090c2bf4f3485cbe4f9c7073c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:19.358436 5317 raft_consensus.cc:1261] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Refusing update from remote peer 47714090c2bf4f3485cbe4f9c7073c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:19.358664 5693 consensus_queue.cc:1035] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c81caff9655422caec120acb4ed1283" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:19.358747 5658 consensus_queue.cc:1035] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4777388a97bf4ba9b4584c71d95e73d9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:19.370914 4831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:19.371027 4831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:19.371678 4831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9825438b68f4a2f89fc5ac17616ee24: Sending DeleteTablet for 3 replicas of tablet 5e13d068be4a45caab62b7f4f7e26210
I20260807 06:33:19.371953 5504 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e13d068be4a45caab62b7f4f7e26210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:19 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:38400
I20260807 06:33:19.372057 5682 tablet_replica.cc:330] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: stopping tablet replica
I20260807 06:33:19.372048 5065 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e13d068be4a45caab62b7f4f7e26210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:19 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:41186
I20260807 06:33:19.372102 5682 raft_consensus.cc:2229] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:19.372139 5282 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e13d068be4a45caab62b7f4f7e26210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:19 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:58648
I20260807 06:33:19.372180 5682 raft_consensus.cc:2258] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:19.372222 5683 tablet_replica.cc:330] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: stopping tablet replica
I20260807 06:33:19.372277 5683 raft_consensus.cc:2229] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:19.372313 5683 raft_consensus.cc:2258] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:19.372399 5684 tablet_replica.cc:330] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: stopping tablet replica
I20260807 06:33:19.372447 5684 raft_consensus.cc:2229] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:19.372450 5682 ts_tablet_manager.cc:1905] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:19.372512 5684 raft_consensus.cc:2258] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:19.372872 5683 ts_tablet_manager.cc:1905] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:19.374135 5683 ts_tablet_manager.cc:1918] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:19.374158 5682 ts_tablet_manager.cc:1918] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:19.374207 5683 log.cc:1196] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/5e13d068be4a45caab62b7f4f7e26210
I20260807 06:33:19.374212 5682 log.cc:1196] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/5e13d068be4a45caab62b7f4f7e26210
I20260807 06:33:19.374441 5683 ts_tablet_manager.cc:1939] T 5e13d068be4a45caab62b7f4f7e26210 P 4777388a97bf4ba9b4584c71d95e73d9: Deleting consensus metadata
I20260807 06:33:19.374449 5682 ts_tablet_manager.cc:1939] T 5e13d068be4a45caab62b7f4f7e26210 P 47714090c2bf4f3485cbe4f9c7073c2f: Deleting consensus metadata
I20260807 06:33:19.374807 4810 catalog_manager.cc:4841] TS 47714090c2bf4f3485cbe4f9c7073c2f (127.2.243.3:44361): tablet 5e13d068be4a45caab62b7f4f7e26210 (table test-table [id=461df4eaac7d464984090f5d25163c87]) successfully deleted
I20260807 06:33:19.374894 4809 catalog_manager.cc:4841] TS 4777388a97bf4ba9b4584c71d95e73d9 (127.2.243.1:46615): tablet 5e13d068be4a45caab62b7f4f7e26210 (table test-table [id=461df4eaac7d464984090f5d25163c87]) successfully deleted
I20260807 06:33:19.376730 5684 ts_tablet_manager.cc:1905] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:19.378180 5684 ts_tablet_manager.cc:1918] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:19.378255 5684 log.cc:1196] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/5e13d068be4a45caab62b7f4f7e26210
I20260807 06:33:19.378470 5684 ts_tablet_manager.cc:1939] T 5e13d068be4a45caab62b7f4f7e26210 P 3c81caff9655422caec120acb4ed1283: Deleting consensus metadata
I20260807 06:33:19.378756 4809 catalog_manager.cc:4841] TS 3c81caff9655422caec120acb4ed1283 (127.2.243.2:41287): tablet 5e13d068be4a45caab62b7f4f7e26210 (table test-table [id=461df4eaac7d464984090f5d25163c87]) successfully deleted
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[4767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:19.386615 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.376892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41198 (local address 127.2.243.1:46615)
0807 06:33:19.376958 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.376961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.377922 (+ 961us) server_negotiation.cc:407] Connection header received
0807 06:33:19.377949 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.377950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.377963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.378069 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:19.378696 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.379210 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.379969 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.380175 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.380247 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.383381 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:19.383389 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:19.383389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:19.383396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:19.385350 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.385725 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.385728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.385728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.385782 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.385985 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.385985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.385985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.386176 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:19.386332 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.386446 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.386538 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 06:33:19.387110 5154 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41198
I20260807 06:33:19.394405 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:19.388193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.2.243.62:40497)
0807 06:33:19.388232 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:19.388233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:19.388633 (+ 400us) server_negotiation.cc:407] Connection header received
0807 06:33:19.388707 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:19.388708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:19.388718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:19.388764 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:19.389292 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.389774 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.390456 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:19.390670 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:19.390742 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:19.391886 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:19.391887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:19.391887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:19.391895 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:19.393294 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.393639 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.393640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.393641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.393701 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:19.393893 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:19.393894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:19.393894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:19.394005 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:19.394112 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:19.394231 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 06:33:19.394307 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:33:19.394610 4831 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44630
I20260807 06:33:19.395171 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4949
I20260807 06:33:19.419303 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5179
I20260807 06:33:19.427927 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5402
I20260807 06:33:19.439492 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4786
2026-08-07T06:33:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2387 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[5734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[5734](info): set up 2 sockets
Aug 07 06:33:19 dist-test-slave-rwrg krb5kdc[5734](info): commencing operation
krb5kdc: starting...
W20260807 06:33:20.745468 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:20 dist-test-slave-rwrg krb5kdc[5734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:20.866611 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:46547
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:46547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:20.946884 5750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:20.947082 5750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:20.947104 5750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:20.948171 5750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:20.948221 5750 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:33:20.948235 5750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:20.948248 5750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:20.948261 5750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:20.948273 5750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:20.949715 5750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:46547
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:46547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:20.949892 5750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:20.950055 5750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:20.955130 5761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:20.955180 5757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:20.955147 5758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:20.955205 5750 server_base.cc:1034] running on GCE node
I20260807 06:33:20.955674 5750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:20.955950 5750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:20.957149 5750 hybrid_clock.cc:648] HybridClock initialized: now 1786084400957149 us; error 49 us; skew 500 ppm
Aug 07 06:33:20 dist-test-slave-rwrg krb5kdc[5734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084400, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:20.959815 5750 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:20.960282 5750 webserver.cc:460] Webserver started at http://127.2.243.62:41285/ using document root <none> and password file <none>
I20260807 06:33:20.960547 5750 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:20.960614 5750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:20.960768 5750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:20.961843 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "aa8dd5c8f41c446db21eebaf72300d66"
format_stamp: "Formatted at 2026-08-07 06:33:20 on dist-test-slave-rwrg"
server_key: "2b4d77c5e73d152bb3abd374caa993c1"
server_key_iv: "80c98cb7ba541ae1fcbff22664297126"
server_key_version: "encryptionkey@0"
I20260807 06:33:20.962196 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "aa8dd5c8f41c446db21eebaf72300d66"
format_stamp: "Formatted at 2026-08-07 06:33:20 on dist-test-slave-rwrg"
server_key: "2b4d77c5e73d152bb3abd374caa993c1"
server_key_iv: "80c98cb7ba541ae1fcbff22664297126"
server_key_version: "encryptionkey@0"
I20260807 06:33:20.963498 5750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:20.964195 5770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:20.964349 5750 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:20.964449 5750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "aa8dd5c8f41c446db21eebaf72300d66"
format_stamp: "Formatted at 2026-08-07 06:33:20 on dist-test-slave-rwrg"
server_key: "2b4d77c5e73d152bb3abd374caa993c1"
server_key_iv: "80c98cb7ba541ae1fcbff22664297126"
server_key_version: "encryptionkey@0"
I20260807 06:33:20.964560 5750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:20.976989 5750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:20.979174 5750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:20.979329 5750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:20.984176 5750 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:46547
I20260807 06:33:20.984289 5869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:46547 every 8 connection(s)
I20260807 06:33:20.984717 5750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:20.985488 5871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:20.987704 5871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Bootstrap starting.
I20260807 06:33:20.988246 5871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:20.988468 5871 log.cc:824] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:20.989284 5871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: No bootstrap required, opened a new log
I20260807 06:33:20.991376 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5750
I20260807 06:33:20.991312 5871 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dd5c8f41c446db21eebaf72300d66" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46547 } }
I20260807 06:33:20.991436 5871 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:20.991478 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:20.991457 5871 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8dd5c8f41c446db21eebaf72300d66, State: Initialized, Role: FOLLOWER
I20260807 06:33:20.991637 5871 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dd5c8f41c446db21eebaf72300d66" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46547 } }
I20260807 06:33:20.991663 3020 external_mini_cluster.cc:1428] Setting key 01675defcd173f019981f95ee083b9eb
I20260807 06:33:20.991720 5871 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:20.991757 5871 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:20.991838 5871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:20.992599 5871 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dd5c8f41c446db21eebaf72300d66" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46547 } }
I20260807 06:33:20.992713 5871 leader_election.cc:302] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa8dd5c8f41c446db21eebaf72300d66; no voters:
I20260807 06:33:20.992867 5871 leader_election.cc:288] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:20.992889 5877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:20.993028 5877 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [term 1 LEADER]: Becoming Leader. State: Replica: aa8dd5c8f41c446db21eebaf72300d66, State: Running, Role: LEADER
I20260807 06:33:20.993112 5877 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dd5c8f41c446db21eebaf72300d66" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46547 } }
I20260807 06:33:20.993157 5871 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:20.993441 5878 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa8dd5c8f41c446db21eebaf72300d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dd5c8f41c446db21eebaf72300d66" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46547 } } }
I20260807 06:33:20.993544 5878 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:20.993440 5880 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa8dd5c8f41c446db21eebaf72300d66. Latest consensus state: current_term: 1 leader_uuid: "aa8dd5c8f41c446db21eebaf72300d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dd5c8f41c446db21eebaf72300d66" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46547 } } }
I20260807 06:33:20.993861 5880 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:20.994138 5884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:20.994714 5884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:20.996946 5884 catalog_manager.cc:1347] Generated new cluster ID: 9cbedddce2324b2795f04f7074749c46
I20260807 06:33:20.997001 5884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:20 dist-test-slave-rwrg krb5kdc[5734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:21.003257 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:20.992139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.2.243.62:46547)
0807 06:33:20.992346 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:20.992355 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:20.992390 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:33:20.992530 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:20.992538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:20.992595 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:20.992743 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:20.993525 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:20.994963 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:20.996036 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:20.996446 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:20.996597 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:20.999319 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:20.999323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:20.999323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:20.999336 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.001458 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.001854 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.001858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.001858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.001996 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.002190 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.002191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.002191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.002499 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.002656 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.002798 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.003053 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260807 06:33:21.006944 5884 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:21.007438 5884 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:21.013468 5884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Generated new TSK 0
I20260807 06:33:21.013651 5884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:21.046880 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:46547
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:21.130887 5906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:21.131099 5906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:21.131124 5906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:21.132298 5906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:21.132340 5906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:21.132361 5906 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:33:21.132372 5906 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:21.132390 5906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:21.133879 5906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:46547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:21.134078 5906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:21.134271 5906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:21.139371 5913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:21.139400 5912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:21.139371 5916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:21.139635 5906 server_base.cc:1034] running on GCE node
I20260807 06:33:21.139845 5906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:21.140077 5906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:21.141240 5906 hybrid_clock.cc:648] HybridClock initialized: now 1786084401141221 us; error 43 us; skew 500 ppm
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:21.143834 5906 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:21.144264 5906 webserver.cc:460] Webserver started at http://127.2.243.1:39267/ using document root <none> and password file <none>
I20260807 06:33:21.144539 5906 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:21.144598 5906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:21.144695 5906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:21.145653 5906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "4942c6bb37474baf8de29784c396644f"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "0f20fba7f97f229761196425c5530b0c"
server_key_iv: "cf3f3e4265b875893ff32d362e6875e6"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.145938 5906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "4942c6bb37474baf8de29784c396644f"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "0f20fba7f97f229761196425c5530b0c"
server_key_iv: "cf3f3e4265b875893ff32d362e6875e6"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.147125 5906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:21.147775 5929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.147919 5906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:21.148007 5906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "4942c6bb37474baf8de29784c396644f"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "0f20fba7f97f229761196425c5530b0c"
server_key_iv: "cf3f3e4265b875893ff32d362e6875e6"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.148072 5906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:21.160665 5906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:21.162535 5906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:21.162671 5906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:21.163156 5906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:21.163199 5906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.163221 5906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:21.163264 5906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.170168 5906 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:41019
I20260807 06:33:21.170248 6104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:41019 every 8 connection(s)
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:21.170542 5906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:21.172254 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5906
I20260807 06:33:21.172361 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:21.172597 3020 external_mini_cluster.cc:1428] Setting key 250ad18dd35508bd4b334e0fef792126
I20260807 06:33:21.174221 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.163737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:50735 (local address 127.2.243.62:46547)
0807 06:33:21.163807 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.163810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.164005 (+ 195us) server_negotiation.cc:407] Connection header received
0807 06:33:21.164527 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.164528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.164546 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.164612 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.165429 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.166252 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.167873 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.168253 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.168359 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.171075 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.171077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.171077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.171087 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.172741 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.173205 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.173206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.173207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.173269 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.173532 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.173532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.173533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.173671 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.173743 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.174022 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.174128 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:33:21.174387 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.163695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:46547 (local address 127.2.243.1:50735)
0807 06:33:21.163873 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.163912 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.164395 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.164683 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.164687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.165035 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:21.165312 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.165319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.166367 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.166368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.167775 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.167778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.168359 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.168360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.168457 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.169052 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:21.169062 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:21.170987 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:21.172825 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.172832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.172835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.173126 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.173344 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.173345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.173346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.173457 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.173747 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:21.173754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:21.173886 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.173984 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.174152 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260807 06:33:21.175191 6105 heartbeater.cc:347] Connected to a master server at 127.2.243.62:46547
I20260807 06:33:21.175278 6105 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:21.175459 6105 heartbeater.cc:511] Master 127.2.243.62:46547 requested a full tablet report, sending...
I20260807 06:33:21.176246 5793 ts_manager.cc:194] Registered new tserver with Master: 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019)
I20260807 06:33:21.176913 5793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:50735
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
I20260807 06:33:21.186693 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.178597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:39471 (local address 127.2.243.62:46547)
0807 06:33:21.178656 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.178658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.180658 (+ 2000us) server_negotiation.cc:407] Connection header received
0807 06:33:21.180731 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.180732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.180742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.180798 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.181108 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.181606 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.182294 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.182498 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.182573 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.183572 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.183574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.183574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.183581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.185471 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.185829 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.185831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.185831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.185898 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.186098 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.186099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.186099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.186246 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.186316 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.186495 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.186597 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:33:21.186709 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.178533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:46547 (local address 127.2.243.1:39471)
0807 06:33:21.180593 (+ 2060us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.180603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.180672 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.180802 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.180803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.180978 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:21.181031 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.181032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.181704 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.181705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.182199 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.182200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.182577 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.182578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.182655 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.183127 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:21.183131 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:21.183427 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:21.185568 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.185570 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.185570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.185742 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.185963 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.185964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.185964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.186016 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.186314 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:21.186315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:21.186364 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.186454 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.186571 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2038,"thread_start_us":70,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:21.200763 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:46547
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:21.285347 6117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:21.285619 6117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:21.285686 6117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:21.286900 6117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:21.286967 6117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:21.287006 6117 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:33:21.287038 6117 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:21.287075 6117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:21.288601 6117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:46547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:21.288851 6117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:21.289086 6117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:21.294167 6125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:21.294186 6126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:21.294215 6129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:21.294457 6117 server_base.cc:1034] running on GCE node
I20260807 06:33:21.294682 6117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:21.294904 6117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:21.296060 6117 hybrid_clock.cc:648] HybridClock initialized: now 1786084401296048 us; error 38 us; skew 500 ppm
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:21.298696 6117 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:21.299142 6117 webserver.cc:460] Webserver started at http://127.2.243.2:36853/ using document root <none> and password file <none>
I20260807 06:33:21.299327 6117 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:21.299366 6117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:21.299453 6117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:21.300410 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "a258348e35404f75ad925d9b5c9f23a0"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "a103e1d299b51e7fb50a9a8942b5d14c"
server_key_iv: "446200a7def1fd0e1a6f8d54d69c93f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.300832 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "a258348e35404f75ad925d9b5c9f23a0"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "a103e1d299b51e7fb50a9a8942b5d14c"
server_key_iv: "446200a7def1fd0e1a6f8d54d69c93f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.302260 6117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:21.303020 6145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.303180 6117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:21.303265 6117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "a258348e35404f75ad925d9b5c9f23a0"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "a103e1d299b51e7fb50a9a8942b5d14c"
server_key_iv: "446200a7def1fd0e1a6f8d54d69c93f3"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.303318 6117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:21.318168 6117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:21.320005 6117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:21.320129 6117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:21.320729 6117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:21.320770 6117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.320791 6117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:21.320804 6117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.328064 6117 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:40979
I20260807 06:33:21.328143 6319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:40979 every 8 connection(s)
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:21.328456 6117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:21.331416 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.321159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:34027 (local address 127.2.243.62:46547)
0807 06:33:21.321232 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.321235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.321484 (+ 249us) server_negotiation.cc:407] Connection header received
0807 06:33:21.322226 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.322228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.322239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.322298 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.323228 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.324056 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.325133 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.325450 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.325525 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.328967 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.328969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.328969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.328982 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.330207 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.330599 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.330600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.330601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.330655 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.330859 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.330859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.330860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.330964 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.331020 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.331291 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.331353 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:33:21.331671 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.321126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:46547 (local address 127.2.243.2:34027)
0807 06:33:21.321347 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.321397 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.322104 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.322401 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.322406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.322802 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:21.323115 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.323122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.324174 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.324175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.325027 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.325029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.325559 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.325560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.325734 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.326862 (+ 1128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:21.326874 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:21.328858 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:21.330284 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.330290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.330292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.330526 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.330714 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.330715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.330715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.330808 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.331039 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:21.331045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:21.331176 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.331268 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.331390 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":117,"threads_started":1}
I20260807 06:33:21.332165 6320 heartbeater.cc:347] Connected to a master server at 127.2.243.62:46547
I20260807 06:33:21.332252 6320 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:21.332458 6320 heartbeater.cc:511] Master 127.2.243.62:46547 requested a full tablet report, sending...
I20260807 06:33:21.332844 5794 ts_manager.cc:194] Registered new tserver with Master: a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979)
I20260807 06:33:21.333235 5794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:34027
I20260807 06:33:21.335866 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6117
I20260807 06:33:21.335983 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:21.336175 3020 external_mini_cluster.cc:1428] Setting key 8b29cbf8b39f34559f20b0a3689ffb66
I20260807 06:33:21.338985 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.333266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:54337 (local address 127.2.243.62:46547)
0807 06:33:21.333344 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.333346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.333405 (+ 59us) server_negotiation.cc:407] Connection header received
0807 06:33:21.333477 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.333478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.333488 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.333544 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.333866 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.334330 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.335017 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.335218 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.335285 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.336221 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.336222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.336223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.336230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.337831 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.338174 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.338175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.338176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.338237 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.338410 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.338411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.338411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.338607 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.338684 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.338875 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.338933 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:33:21.339083 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.333208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:46547 (local address 127.2.243.2:54337)
0807 06:33:21.333354 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.333364 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.333420 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.333549 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.333549 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.333726 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:21.333783 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.333785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.334429 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.334430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.334929 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.334931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.335285 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.335286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.335365 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.335773 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:21.335778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:21.336140 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:21.337924 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.337925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.337925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.338108 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.338301 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.338302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.338302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.338353 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.338684 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:21.338685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:21.338752 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.338830 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.338957 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:21.362947 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:46547
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:21.445541 6335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:21.445744 6335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:21.445768 6335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:21.446823 6335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:21.446856 6335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:21.446875 6335 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:33:21.446887 6335 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:21.446902 6335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:21.448312 6335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:46547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:21.448485 6335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:21.448722 6335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:21.453743 6345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:21.453920 6344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:21.453961 6347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:21.454015 6335 server_base.cc:1034] running on GCE node
I20260807 06:33:21.454391 6335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:21.454639 6335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:21.455790 6335 hybrid_clock.cc:648] HybridClock initialized: now 1786084401455773 us; error 34 us; skew 500 ppm
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:21.458634 6335 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:21.459090 6335 webserver.cc:460] Webserver started at http://127.2.243.3:34837/ using document root <none> and password file <none>
I20260807 06:33:21.459321 6335 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:21.459384 6335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:21.459539 6335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:21.460590 6335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "3d2e2d5c73174969b33aa85c6430f25f"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "4bfaf1681da999a4778eb8b98758487e"
server_key_iv: "c71b4aa93f369ba4fe2dbb14f3b85731"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.460934 6335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "3d2e2d5c73174969b33aa85c6430f25f"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "4bfaf1681da999a4778eb8b98758487e"
server_key_iv: "c71b4aa93f369ba4fe2dbb14f3b85731"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.462343 6335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:21.463102 6356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.463253 6335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:21.463363 6335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "3d2e2d5c73174969b33aa85c6430f25f"
format_stamp: "Formatted at 2026-08-07 06:33:21 on dist-test-slave-rwrg"
server_key: "4bfaf1681da999a4778eb8b98758487e"
server_key_iv: "c71b4aa93f369ba4fe2dbb14f3b85731"
server_key_version: "encryptionkey@0"
I20260807 06:33:21.463447 6335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:21.486471 6335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:21.488896 6335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:21.489097 6335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:21.489691 6335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:21.489746 6335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.489789 6335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:21.489814 6335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:21.497521 6335 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:45527
I20260807 06:33:21.497870 6335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:21.497906 6542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:45527 every 8 connection(s)
I20260807 06:33:21.498174 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6335
I20260807 06:33:21.498278 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:21.498497 3020 external_mini_cluster.cc:1428] Setting key 61d0db423783b38e5da49293ad726254
I20260807 06:33:21.500473 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.490140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:56779 (local address 127.2.243.62:46547)
0807 06:33:21.490209 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.490212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.490521 (+ 309us) server_negotiation.cc:407] Connection header received
0807 06:33:21.491190 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.491191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.491201 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.491249 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.492158 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.492906 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.494171 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.494480 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.494564 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.497070 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.497072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.497073 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.497081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.499266 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.499725 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.499726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.499726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.499780 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.499952 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.499952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.499952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.500061 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.500109 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.500351 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.500412 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:33:21.500787 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.490107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:46547 (local address 127.2.243.3:56779)
0807 06:33:21.490320 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.490374 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.491079 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.491330 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.491335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.491732 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:21.492045 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.492053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.493025 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.493027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.494067 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.494070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.494593 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.494595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.494740 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.495268 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:21.495276 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:21.496962 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:21.499376 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.499384 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.499386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.499660 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.499823 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.499823 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.499824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.499906 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.500115 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:21.500119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:21.500244 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.500327 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.500445 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260807 06:33:21.501252 6544 heartbeater.cc:347] Connected to a master server at 127.2.243.62:46547
I20260807 06:33:21.501339 6544 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:21.501529 6544 heartbeater.cc:511] Master 127.2.243.62:46547 requested a full tablet report, sending...
I20260807 06:33:21.501915 5793 ts_manager.cc:194] Registered new tserver with Master: 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527)
I20260807 06:33:21.502384 5793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:56779
I20260807 06:33:21.507649 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.502328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:42481 (local address 127.2.243.62:46547)
0807 06:33:21.502371 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.502373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.502499 (+ 126us) server_negotiation.cc:407] Connection header received
0807 06:33:21.502595 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.502596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.502606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.502652 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.503037 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.503588 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.504323 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.504560 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.504621 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.505540 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.505541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.505542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.505548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.506736 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.507042 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.507043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.507043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.507104 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.507279 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.507279 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.507279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.507370 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.507419 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.507538 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.507595 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:21.507660 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.502273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:46547 (local address 127.2.243.3:42481)
0807 06:33:21.502424 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.502434 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.502532 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.502665 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.502666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.502883 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:21.502952 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.502954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.503678 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.503678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.504237 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.504239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.504632 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.504633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.504699 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.505099 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:21.505103 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:21.505398 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:21.506804 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.506805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.506805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.506965 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.507169 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.507170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.507170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.507229 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.507421 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:21.507422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:21.507465 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.507521 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.507596 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260807 06:33:21.510201 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:21.534400 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.527143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.2.243.62:46547)
0807 06:33:21.527210 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.527212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.527222 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:21.527254 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.527254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.527263 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.527325 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.527926 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.528407 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.529158 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.529356 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.529431 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.531860 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.531862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.531862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.531869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.533301 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.533633 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.533634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.533635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.533689 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.533861 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.533862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.533862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.533964 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.534058 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.534269 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.534335 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:33:21.536294 5793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:21.538053 5793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:21.544796 6571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.540375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:41019 (local address 127.0.0.1:33284)
0807 06:33:21.540857 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.540899 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.541159 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.541369 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.541369 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.541372 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:21.541426 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.541428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.542329 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.542330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.543516 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.543518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.544488 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.544514 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.544579 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.544585 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.544641 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.544709 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":21,"thread_start_us":77,"threads_started":1}
I20260807 06:33:21.544847 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.540598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.2.243.3:45527)
0807 06:33:21.540798 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.540803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.540988 (+ 185us) server_negotiation.cc:407] Connection header received
0807 06:33:21.541073 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.541076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.541094 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.541184 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:21.541447 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.542608 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.543847 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.544336 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.544423 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.544487 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.544650 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.544695 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260807 06:33:21.544958 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.540420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:40979 (local address 127.0.0.1:48242)
0807 06:33:21.540879 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.540918 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.541000 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.541461 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.541462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.541464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:21.541496 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.541497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.542750 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.542751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.543733 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.543736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.544704 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.544705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.544779 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.544783 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.544818 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.544855 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260807 06:33:21.545234 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.540703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.2.243.1:41019)
0807 06:33:21.541150 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.541155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.541167 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:21.541264 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.541267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.541285 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.541360 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:21.541514 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.542218 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.543611 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.544365 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.544698 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.544766 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.544794 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.544840 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":83,"threads_started":1}
I20260807 06:33:21.545295 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.540375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45527 (local address 127.0.0.1:57908)
0807 06:33:21.540857 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.540902 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.540976 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.541274 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.541277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.541280 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:21.541350 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.541352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.542708 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.542709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.543761 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.543762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.544423 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.544424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.544539 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.544556 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.544634 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.545216 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":26,"thread_start_us":66,"threads_started":1}
I20260807 06:33:21.545414 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.540698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.2.243.2:40979)
0807 06:33:21.541272 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.541276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.541286 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:21.541325 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.541327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.541343 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.541462 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:21.541559 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.542632 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.543847 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.544597 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.545165 (+ 568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.545236 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.545264 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.545309 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":92,"threads_started":1}
I20260807 06:33:21.545945 6009 tablet_service.cc:1467] Processing CreateTablet for tablet 0092cc9c7f014d5e830c939075cd91d0 (DEFAULT_TABLE table=test-table [id=12eb31ee16b5455587ce3b023c3a6a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:21.545998 6426 tablet_service.cc:1467] Processing CreateTablet for tablet 0092cc9c7f014d5e830c939075cd91d0 (DEFAULT_TABLE table=test-table [id=12eb31ee16b5455587ce3b023c3a6a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:21.546002 6216 tablet_service.cc:1467] Processing CreateTablet for tablet 0092cc9c7f014d5e830c939075cd91d0 (DEFAULT_TABLE table=test-table [id=12eb31ee16b5455587ce3b023c3a6a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:21.546242 6009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0092cc9c7f014d5e830c939075cd91d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:21.546339 6216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0092cc9c7f014d5e830c939075cd91d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:21.546352 6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0092cc9c7f014d5e830c939075cd91d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:21.549067 6580 tablet_bootstrap.cc:492] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Bootstrap starting.
I20260807 06:33:21.549631 6580 tablet_bootstrap.cc:654] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:21.549870 6580 log.cc:824] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:21.550372 6581 tablet_bootstrap.cc:492] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Bootstrap starting.
I20260807 06:33:21.551134 6581 tablet_bootstrap.cc:654] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:21.551394 6584 tablet_bootstrap.cc:492] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Bootstrap starting.
I20260807 06:33:21.551939 6584 tablet_bootstrap.cc:654] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:21.552239 6584 log.cc:824] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:21.552650 6581 log.cc:824] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:21.553056 6580 tablet_bootstrap.cc:492] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: No bootstrap required, opened a new log
I20260807 06:33:21.553090 6584 tablet_bootstrap.cc:492] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: No bootstrap required, opened a new log
I20260807 06:33:21.553140 6580 ts_tablet_manager.cc:1397] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:33:21.553154 6584 ts_tablet_manager.cc:1397] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:33:21.553467 6581 tablet_bootstrap.cc:492] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: No bootstrap required, opened a new log
I20260807 06:33:21.553529 6581 ts_tablet_manager.cc:1397] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:33:21.554643 6584 raft_consensus.cc:348] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.554787 6584 raft_consensus.cc:374] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:21.554831 6584 raft_consensus.cc:729] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a258348e35404f75ad925d9b5c9f23a0, State: Initialized, Role: FOLLOWER
I20260807 06:33:21.554942 6584 consensus_queue.cc:260] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.555230 6584 ts_tablet_manager.cc:1428] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:21.555255 6320 heartbeater.cc:503] Master 127.2.243.62:46547 was elected leader, sending a full tablet report...
I20260807 06:33:21.555430 6581 raft_consensus.cc:348] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.555576 6581 raft_consensus.cc:374] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:21.555620 6581 raft_consensus.cc:729] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d2e2d5c73174969b33aa85c6430f25f, State: Initialized, Role: FOLLOWER
I20260807 06:33:21.555639 6580 raft_consensus.cc:348] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.555812 6580 raft_consensus.cc:374] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:21.555850 6580 raft_consensus.cc:729] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4942c6bb37474baf8de29784c396644f, State: Initialized, Role: FOLLOWER
I20260807 06:33:21.555797 6581 consensus_queue.cc:260] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.555975 6580 consensus_queue.cc:260] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.556139 6581 ts_tablet_manager.cc:1428] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:33:21.556275 6544 heartbeater.cc:503] Master 127.2.243.62:46547 was elected leader, sending a full tablet report...
I20260807 06:33:21.556274 6580 ts_tablet_manager.cc:1428] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:33:21.556412 6105 heartbeater.cc:503] Master 127.2.243.62:46547 was elected leader, sending a full tablet report...
W20260807 06:33:21.579087 6322 tablet.cc:2369] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:33:21.671334 6106 tablet.cc:2369] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:33:21.748478 6547 tablet.cc:2369] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:21.793009 6591 raft_consensus.cc:482] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:21.793219 6591 raft_consensus.cc:504] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.793645 6591 leader_election.cc:288] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527), a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979)
I20260807 06:33:21.797791 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.793969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:58997 (local address 127.2.243.2:40979)
0807 06:33:21.794030 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.794033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.794097 (+ 64us) server_negotiation.cc:407] Connection header received
0807 06:33:21.794212 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.794212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.794237 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.794323 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:21.794513 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.795273 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.796877 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.797434 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.797513 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.797522 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.797683 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.797722 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:33:21.797791 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.793969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:49317 (local address 127.2.243.3:45527)
0807 06:33:21.794030 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.794033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.794106 (+ 73us) server_negotiation.cc:407] Connection header received
0807 06:33:21.794258 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.794259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.794272 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.794331 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:21.794507 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.795273 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.796882 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.797434 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.797510 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.797522 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.797683 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.797722 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:33:21.797838 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.793850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:40979 (local address 127.2.243.1:58997)
0807 06:33:21.794028 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.794040 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.794144 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.794347 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.794348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.794350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:21.794431 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.794432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.795366 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.795367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.796784 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.796788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.797519 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.797520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.797598 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.797603 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.797666 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.797718 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":102,"thread_start_us":77,"threads_started":1}
I20260807 06:33:21.797840 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.793850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45527 (local address 127.2.243.1:49317)
0807 06:33:21.793995 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.794040 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.794136 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.794347 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.794347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.794351 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:21.794428 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.794430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.795366 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.795367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.796785 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.796788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.797546 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.797547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.797607 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.797609 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.797666 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.797718 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":80,"thread_start_us":75,"threads_started":1}
I20260807 06:33:21.798166 6250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0092cc9c7f014d5e830c939075cd91d0" candidate_uuid: "4942c6bb37474baf8de29784c396644f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a258348e35404f75ad925d9b5c9f23a0" is_pre_election: true
I20260807 06:33:21.798341 6250 raft_consensus.cc:2454] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4942c6bb37474baf8de29784c396644f in term 0.
I20260807 06:33:21.798365 6453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0092cc9c7f014d5e830c939075cd91d0" candidate_uuid: "4942c6bb37474baf8de29784c396644f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d2e2d5c73174969b33aa85c6430f25f" is_pre_election: true
I20260807 06:33:21.798499 6453 raft_consensus.cc:2454] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4942c6bb37474baf8de29784c396644f in term 0.
I20260807 06:33:21.798748 5936 leader_election.cc:302] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, 4942c6bb37474baf8de29784c396644f; no voters:
I20260807 06:33:21.798938 6591 raft_consensus.cc:2781] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:21.799026 6591 raft_consensus.cc:482] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:21.799064 6591 raft_consensus.cc:3037] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:21.799753 6591 raft_consensus.cc:504] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.799901 6591 leader_election.cc:288] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [CANDIDATE]: Term 1 election: Requested vote from peers 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527), a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979)
I20260807 06:33:21.800067 6453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0092cc9c7f014d5e830c939075cd91d0" candidate_uuid: "4942c6bb37474baf8de29784c396644f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d2e2d5c73174969b33aa85c6430f25f"
I20260807 06:33:21.800097 6250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0092cc9c7f014d5e830c939075cd91d0" candidate_uuid: "4942c6bb37474baf8de29784c396644f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a258348e35404f75ad925d9b5c9f23a0"
I20260807 06:33:21.800159 6250 raft_consensus.cc:3037] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:21.800150 6453 raft_consensus.cc:3037] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:21.801096 6453 raft_consensus.cc:2454] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4942c6bb37474baf8de29784c396644f in term 1.
I20260807 06:33:21.801096 6250 raft_consensus.cc:2454] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4942c6bb37474baf8de29784c396644f in term 1.
I20260807 06:33:21.801266 5936 leader_election.cc:302] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, 4942c6bb37474baf8de29784c396644f; no voters:
I20260807 06:33:21.801376 6591 raft_consensus.cc:2781] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:21.801496 6591 raft_consensus.cc:686] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 1 LEADER]: Becoming Leader. State: Replica: 4942c6bb37474baf8de29784c396644f, State: Running, Role: LEADER
I20260807 06:33:21.801595 6591 consensus_queue.cc:237] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } }
I20260807 06:33:21.802505 5794 catalog_manager.cc:5495] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4942c6bb37474baf8de29784c396644f (127.2.243.1). New cstate: current_term: 1 leader_uuid: "4942c6bb37474baf8de29784c396644f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } health_report { overall_health: UNKNOWN } } }
Aug 07 06:33:21 dist-test-slave-rwrg krb5kdc[5734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084400, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:21.856241 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.849279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:46547 (local address 127.0.0.1:49134)
0807 06:33:21.849424 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.849435 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.849510 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.849643 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.849644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.849950 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:21.850001 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.850003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.850722 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.850723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.851268 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.851270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.851603 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.851604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.851651 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.852144 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:21.852148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:21.853577 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:21.855179 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.855188 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.855190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.855424 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.855660 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:21.855661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:21.855662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:21.855725 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:21.855957 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:21.855964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:21.856014 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.856074 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.856166 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260807 06:33:21.856309 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.849362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.2.243.62:46547)
0807 06:33:21.849441 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.849443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.849496 (+ 53us) server_negotiation.cc:407] Connection header received
0807 06:33:21.849563 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.849563 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.849572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.849626 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:21.850064 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.850651 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.851332 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.851509 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.851591 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.853674 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:21.853676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:21.853676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:21.853683 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:21.855089 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.855537 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.855537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.855538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.855589 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:21.855789 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:21.855790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:21.855790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:21.855905 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:21.855955 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.856102 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.856228 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":47}
I20260807 06:33:21.858481 5794 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:49134:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:33:21.858660 5794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:21.860152 6009 tablet_service.cc:1467] Processing CreateTablet for tablet 956972fe3cf940d5b61bb1258661ff3f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b493b002382741a4a2d82849b7331e13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:21.860129 6216 tablet_service.cc:1467] Processing CreateTablet for tablet 956972fe3cf940d5b61bb1258661ff3f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b493b002382741a4a2d82849b7331e13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:21.860193 6426 tablet_service.cc:1467] Processing CreateTablet for tablet 956972fe3cf940d5b61bb1258661ff3f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b493b002382741a4a2d82849b7331e13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:21.860327 6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 956972fe3cf940d5b61bb1258661ff3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:21.860335 6009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 956972fe3cf940d5b61bb1258661ff3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:21.860328 6216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 956972fe3cf940d5b61bb1258661ff3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:21.861514 6580 tablet_bootstrap.cc:492] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: Bootstrap starting.
I20260807 06:33:21.861881 6581 tablet_bootstrap.cc:492] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: Bootstrap starting.
I20260807 06:33:21.861945 6580 tablet_bootstrap.cc:654] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:21.862504 6581 tablet_bootstrap.cc:654] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:21.862849 6584 tablet_bootstrap.cc:492] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: Bootstrap starting.
I20260807 06:33:21.863022 6580 tablet_bootstrap.cc:492] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: No bootstrap required, opened a new log
I20260807 06:33:21.863085 6580 ts_tablet_manager.cc:1397] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:21.863245 6584 tablet_bootstrap.cc:654] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:21.863252 6580 raft_consensus.cc:348] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.863339 6580 raft_consensus.cc:374] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:21.863369 6580 raft_consensus.cc:729] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4942c6bb37474baf8de29784c396644f, State: Initialized, Role: FOLLOWER
I20260807 06:33:21.863423 6580 consensus_queue.cc:260] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.863516 6580 ts_tablet_manager.cc:1428] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:21.863585 6591 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:21.863695 6591 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:21.863878 6581 tablet_bootstrap.cc:492] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: No bootstrap required, opened a new log
I20260807 06:33:21.863963 6581 ts_tablet_manager.cc:1397] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:21.864128 6581 raft_consensus.cc:348] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.864189 6581 raft_consensus.cc:374] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:21.864202 6581 raft_consensus.cc:729] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d2e2d5c73174969b33aa85c6430f25f, State: Initialized, Role: FOLLOWER
I20260807 06:33:21.864281 6581 consensus_queue.cc:260] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.864387 6581 ts_tablet_manager.cc:1428] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:21.864455 6590 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:21.864562 6590 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:21.864723 6584 tablet_bootstrap.cc:492] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: No bootstrap required, opened a new log
I20260807 06:33:21.864789 6584 ts_tablet_manager.cc:1397] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:21.864933 6584 raft_consensus.cc:348] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.865018 6584 raft_consensus.cc:374] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:21.865041 6584 raft_consensus.cc:729] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a258348e35404f75ad925d9b5c9f23a0, State: Initialized, Role: FOLLOWER
I20260807 06:33:21.865092 6584 consensus_queue.cc:260] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.865206 6584 ts_tablet_manager.cc:1428] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:21.865242 6589 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:21.865338 6589 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:21.974958 6590 raft_consensus.cc:482] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:21.975066 6590 raft_consensus.cc:504] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.975495 6590 leader_election.cc:288] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979)
I20260807 06:33:21.978608 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.975812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:46223 (local address 127.2.243.1:41019)
0807 06:33:21.975860 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.975862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.975874 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:21.975962 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.975963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.975981 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.976050 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:21.976204 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.976727 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.977852 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.978231 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.978311 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.978322 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.978491 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.978537 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:33:21.978650 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.975743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:41019 (local address 127.2.243.3:46223)
0807 06:33:21.975812 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.975826 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.975877 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.976054 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.976055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.976057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:21.976133 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.976134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.976806 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.976806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.977779 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.977782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.978322 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.978323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.978417 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.978421 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.978484 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.978526 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":13}
I20260807 06:33:21.978654 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.975923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:44199 (local address 127.2.243.2:40979)
0807 06:33:21.975969 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:21.975971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:21.975979 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:21.976039 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:21.976040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:21.976049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:21.976118 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:21.976269 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.976751 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.977896 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.978280 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.978358 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.978370 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:21.978547 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:33:21.978586 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:33:21.978675 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:21.975768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:40979 (local address 127.2.243.3:44199)
0807 06:33:21.975918 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:21.975929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:21.975984 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:21.976124 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:21.976125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:21.976127 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:21.976200 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.976201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.976837 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.976838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.977814 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:21.977817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:21.978379 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:21.978380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:21.978446 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:21.978450 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:21.978534 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:33:21.978573 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":39,"thread_start_us":91,"threads_started":1}
I20260807 06:33:21.978888 6250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956972fe3cf940d5b61bb1258661ff3f" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a258348e35404f75ad925d9b5c9f23a0" is_pre_election: true
I20260807 06:33:21.978991 6250 raft_consensus.cc:2454] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 0.
I20260807 06:33:21.979162 6364 leader_election.cc:302] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, a258348e35404f75ad925d9b5c9f23a0; no voters:
I20260807 06:33:21.979149 6044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956972fe3cf940d5b61bb1258661ff3f" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f" is_pre_election: true
I20260807 06:33:21.979264 6044 raft_consensus.cc:2454] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 0.
I20260807 06:33:21.979300 6590 raft_consensus.cc:2781] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:21.979337 6590 raft_consensus.cc:482] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:21.979369 6590 raft_consensus.cc:3037] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:21.980136 6590 raft_consensus.cc:504] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.980284 6590 leader_election.cc:288] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 election: Requested vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979)
I20260807 06:33:21.980414 6044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956972fe3cf940d5b61bb1258661ff3f" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f"
I20260807 06:33:21.980458 6250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956972fe3cf940d5b61bb1258661ff3f" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a258348e35404f75ad925d9b5c9f23a0"
I20260807 06:33:21.980548 6044 raft_consensus.cc:3037] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:21.980574 6250 raft_consensus.cc:3037] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:21.981211 6044 raft_consensus.cc:2454] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 1.
I20260807 06:33:21.981295 6250 raft_consensus.cc:2454] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 1.
I20260807 06:33:21.981400 6359 leader_election.cc:302] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, 4942c6bb37474baf8de29784c396644f; no voters:
I20260807 06:33:21.981501 6590 raft_consensus.cc:2781] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:21.981671 6590 raft_consensus.cc:686] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [term 1 LEADER]: Becoming Leader. State: Replica: 3d2e2d5c73174969b33aa85c6430f25f, State: Running, Role: LEADER
I20260807 06:33:21.981778 6590 consensus_queue.cc:237] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:21.982205 6616 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: TxnStatusTablet state changed. Reason: New leader 3d2e2d5c73174969b33aa85c6430f25f. Latest consensus state: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:21.982295 6616 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:21.982487 6618 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:33:21.982620 6618 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:33:21.982606 5794 catalog_manager.cc:5495] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3). New cstate: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } health_report { overall_health: HEALTHY } } }
I20260807 06:33:22.011962 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.009257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45527 (local address 127.0.0.1:57914)
0807 06:33:22.009343 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:22.009354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:22.009393 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:22.009615 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:22.009615 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:22.009617 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:22.009667 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.009669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.010313 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.010314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.010929 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.010931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.011382 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.011384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.011462 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.011477 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.011770 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.011781 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 06:33:22.011838 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:33:22.011888 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 06:33:22.011945 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.009390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.2.243.3:45527)
0807 06:33:22.009452 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:22.009454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:22.009463 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:22.009499 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:22.009500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:22.009521 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:22.009605 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:22.009748 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.010222 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.011095 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.011305 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.011380 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.011608 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.011701 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.011789 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:22.011843 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:33:22.011865 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:33:22.013769 6250 raft_consensus.cc:1261] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Refusing update from remote peer 3d2e2d5c73174969b33aa85c6430f25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:22.013828 6044 raft_consensus.cc:1261] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Refusing update from remote peer 3d2e2d5c73174969b33aa85c6430f25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:22.014082 6590 consensus_queue.cc:1035] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:22.014207 6616 consensus_queue.cc:1035] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:22.015328 6591 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: TxnStatusTablet state changed. Reason: New leader 3d2e2d5c73174969b33aa85c6430f25f. Latest consensus state: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:22.015437 6591 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:22.015466 6589 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: TxnStatusTablet state changed. Reason: New leader 3d2e2d5c73174969b33aa85c6430f25f. Latest consensus state: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:22.015568 6589 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:22.016548 6616 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:22.016629 6616 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:22.016772 6589 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:22.016866 6589 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P a258348e35404f75ad925d9b5c9f23a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:22.017082 6590 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:22.017165 6590 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P 3d2e2d5c73174969b33aa85c6430f25f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:22.017424 6591 tablet_replica.cc:439] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } } }
I20260807 06:33:22.017517 6591 tablet_replica.cc:442] T 956972fe3cf940d5b61bb1258661ff3f P 4942c6bb37474baf8de29784c396644f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:22.017530 6623 mvcc.cc:205] Tried to move back new op lower bound from 7315801710645972992 to 7315801710517940224. Current Snapshot: MvccSnapshot[applied={T|T < 7315801710645972992}]
I20260807 06:33:22.017050 6622 mvcc.cc:205] Tried to move back new op lower bound from 7315801710645972992 to 7315801710517940224. Current Snapshot: MvccSnapshot[applied={T|T < 7315801710645972992}]
I20260807 06:33:22.018085 6621 mvcc.cc:205] Tried to move back new op lower bound from 7315801710645972992 to 7315801710517940224. Current Snapshot: MvccSnapshot[applied={T|T < 7315801710645972992}]
I20260807 06:33:22.022838 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.020187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.2.243.1:41019)
0807 06:33:22.020223 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:22.020225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:22.020234 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:22.020291 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:22.020292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:22.020305 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:22.020355 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:22.020476 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.021084 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.022182 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.022359 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.022415 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.022581 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.022663 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.022719 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:22.022777 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 06:33:22.022793 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:33:22.027719 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.025243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:34461 (local address 127.2.243.3:45527)
0807 06:33:22.025279 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:22.025281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:22.025440 (+ 159us) server_negotiation.cc:407] Connection header received
0807 06:33:22.025524 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:22.025524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:22.025535 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:22.025586 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:22.025720 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.026165 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.027035 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.027220 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.027308 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.027487 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.027520 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.027577 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:22.027637 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:33:22.027654 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:22.027757 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.025176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45527 (local address 127.2.243.1:34461)
0807 06:33:22.025353 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:22.025362 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:22.025465 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:22.025604 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:22.025605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:22.025606 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:22.025668 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.025669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.026244 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.026245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.026952 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.026954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.027311 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.027312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.027372 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.027390 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.027579 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.027582 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:22.027633 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:33:22.027671 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260807 06:33:22.030064 6250 raft_consensus.cc:1261] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Refusing update from remote peer 4942c6bb37474baf8de29784c396644f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:22.030110 6453 raft_consensus.cc:1261] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Refusing update from remote peer 4942c6bb37474baf8de29784c396644f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:22.030376 6595 consensus_queue.cc:1035] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:22.030525 6591 consensus_queue.cc:1035] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:22.039160 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.036674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:53825 (local address 127.2.243.1:41019)
0807 06:33:22.036714 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:22.036715 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:22.036772 (+ 57us) server_negotiation.cc:407] Connection header received
0807 06:33:22.036841 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:22.036842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:22.036852 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:22.036891 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:22.037056 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.037511 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.038451 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.038665 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.038732 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.038914 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.038945 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.038993 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:22.039073 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 06:33:22.039091 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:33:22.039182 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.036495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:41019 (local address 127.2.243.3:53825)
0807 06:33:22.036721 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:22.036731 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:22.036788 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:22.036906 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:22.036906 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:22.036908 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:22.036976 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.036977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.037601 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.037602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.038366 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.038369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.038733 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.038734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.038796 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.038816 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.039014 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.039016 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:22.039061 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 06:33:22.039102 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260807 06:33:22.048352 5794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:22.048465 5794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:22.049183 5794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Sending DeleteTablet for 3 replicas of tablet 0092cc9c7f014d5e830c939075cd91d0
I20260807 06:33:22.049527 6426 tablet_service.cc:1514] Processing DeleteTablet for tablet 0092cc9c7f014d5e830c939075cd91d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:22 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:57908
I20260807 06:33:22.049594 6009 tablet_service.cc:1514] Processing DeleteTablet for tablet 0092cc9c7f014d5e830c939075cd91d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:22 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:33284
I20260807 06:33:22.049945 6216 tablet_service.cc:1514] Processing DeleteTablet for tablet 0092cc9c7f014d5e830c939075cd91d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:22 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:48242
I20260807 06:33:22.049952 5794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:22.050092 5794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:22.051373 6216 tablet_service.cc:1467] Processing CreateTablet for tablet 3c05b9aa1d31459699b4f4e7e36ac442 (DEFAULT_TABLE table=test-table [id=515e41eb6d2649818ff29870333c477a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:22.051471 6009 tablet_service.cc:1467] Processing CreateTablet for tablet 3c05b9aa1d31459699b4f4e7e36ac442 (DEFAULT_TABLE table=test-table [id=515e41eb6d2649818ff29870333c477a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:22.051517 6216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c05b9aa1d31459699b4f4e7e36ac442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:22.051580 6009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c05b9aa1d31459699b4f4e7e36ac442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:22.052989 6580 tablet_bootstrap.cc:492] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: Bootstrap starting.
I20260807 06:33:22.053071 6584 tablet_bootstrap.cc:492] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: Bootstrap starting.
I20260807 06:33:22.053471 6584 tablet_bootstrap.cc:654] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:22.053471 6580 tablet_bootstrap.cc:654] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:22.054278 6426 tablet_service.cc:1467] Processing CreateTablet for tablet 3c05b9aa1d31459699b4f4e7e36ac442 (DEFAULT_TABLE table=test-table [id=515e41eb6d2649818ff29870333c477a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:22.054430 6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c05b9aa1d31459699b4f4e7e36ac442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:22.055395 6581 tablet_bootstrap.cc:492] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: Bootstrap starting.
I20260807 06:33:22.055666 6654 tablet_replica.cc:330] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: stopping tablet replica
I20260807 06:33:22.055721 6581 tablet_bootstrap.cc:654] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:22.055774 6654 raft_consensus.cc:2229] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:22.055894 6654 raft_consensus.cc:2258] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:22.050477 6656 tablet_replica.cc:330] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: stopping tablet replica
I20260807 06:33:22.056080 6656 raft_consensus.cc:2229] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:22.056209 6656 raft_consensus.cc:2258] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:22.056696 6654 ts_tablet_manager.cc:1905] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:22.056847 6581 tablet_bootstrap.cc:492] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: No bootstrap required, opened a new log
I20260807 06:33:22.056890 6581 ts_tablet_manager.cc:1397] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:22.057044 6581 raft_consensus.cc:348] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.057120 6581 raft_consensus.cc:374] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:22.057145 6581 raft_consensus.cc:729] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d2e2d5c73174969b33aa85c6430f25f, State: Initialized, Role: FOLLOWER
I20260807 06:33:22.057186 6581 consensus_queue.cc:260] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.057287 6581 ts_tablet_manager.cc:1428] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:22.057548 6656 ts_tablet_manager.cc:1905] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:22.058107 6652 tablet_replica.cc:330] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: stopping tablet replica
I20260807 06:33:22.058208 6652 raft_consensus.cc:2229] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:22.058280 6652 raft_consensus.cc:2258] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:22.058840 6652 ts_tablet_manager.cc:1905] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:22.059090 6656 ts_tablet_manager.cc:1918] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:22.059229 6656 log.cc:1196] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/0092cc9c7f014d5e830c939075cd91d0
I20260807 06:33:22.059585 6656 ts_tablet_manager.cc:1939] T 0092cc9c7f014d5e830c939075cd91d0 P a258348e35404f75ad925d9b5c9f23a0: Deleting consensus metadata
I20260807 06:33:22.060074 5774 catalog_manager.cc:4841] TS a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979): tablet 0092cc9c7f014d5e830c939075cd91d0 (table test-table [id=12eb31ee16b5455587ce3b023c3a6a19]) successfully deleted
I20260807 06:33:22.060340 6654 ts_tablet_manager.cc:1918] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:22.060420 6654 log.cc:1196] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/0092cc9c7f014d5e830c939075cd91d0
I20260807 06:33:22.060652 6652 ts_tablet_manager.cc:1918] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:22.060719 6654 ts_tablet_manager.cc:1939] T 0092cc9c7f014d5e830c939075cd91d0 P 4942c6bb37474baf8de29784c396644f: Deleting consensus metadata
I20260807 06:33:22.060729 6652 log.cc:1196] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/0092cc9c7f014d5e830c939075cd91d0
I20260807 06:33:22.061012 6652 ts_tablet_manager.cc:1939] T 0092cc9c7f014d5e830c939075cd91d0 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting consensus metadata
I20260807 06:33:22.061111 5771 catalog_manager.cc:4841] TS 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019): tablet 0092cc9c7f014d5e830c939075cd91d0 (table test-table [id=12eb31ee16b5455587ce3b023c3a6a19]) successfully deleted
I20260807 06:33:22.061403 5776 catalog_manager.cc:4841] TS 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527): tablet 0092cc9c7f014d5e830c939075cd91d0 (table test-table [id=12eb31ee16b5455587ce3b023c3a6a19]) successfully deleted
I20260807 06:33:22.061522 6580 tablet_bootstrap.cc:492] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: No bootstrap required, opened a new log
I20260807 06:33:22.061578 6580 ts_tablet_manager.cc:1397] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260807 06:33:22.061724 6580 raft_consensus.cc:348] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.061800 6580 raft_consensus.cc:374] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:22.061820 6580 raft_consensus.cc:729] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4942c6bb37474baf8de29784c396644f, State: Initialized, Role: FOLLOWER
I20260807 06:33:22.061868 6580 consensus_queue.cc:260] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.061957 6580 ts_tablet_manager.cc:1428] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:22.062438 6584 tablet_bootstrap.cc:492] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: No bootstrap required, opened a new log
I20260807 06:33:22.062487 6584 ts_tablet_manager.cc:1397] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260807 06:33:22.062642 6584 raft_consensus.cc:348] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.062712 6584 raft_consensus.cc:374] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:22.062734 6584 raft_consensus.cc:729] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a258348e35404f75ad925d9b5c9f23a0, State: Initialized, Role: FOLLOWER
I20260807 06:33:22.062780 6584 consensus_queue.cc:260] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.062879 6584 ts_tablet_manager.cc:1428] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:22.080653 6616 raft_consensus.cc:482] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:22.080750 6616 raft_consensus.cc:504] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.080907 6616 leader_election.cc:288] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979)
I20260807 06:33:22.081089 6044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05b9aa1d31459699b4f4e7e36ac442" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f" is_pre_election: true
I20260807 06:33:22.081179 6044 raft_consensus.cc:2454] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 0.
I20260807 06:33:22.081172 6250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05b9aa1d31459699b4f4e7e36ac442" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a258348e35404f75ad925d9b5c9f23a0" is_pre_election: true
I20260807 06:33:22.081252 6250 raft_consensus.cc:2454] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 0.
I20260807 06:33:22.081317 6359 leader_election.cc:302] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, 4942c6bb37474baf8de29784c396644f; no voters:
I20260807 06:33:22.081396 6616 raft_consensus.cc:2781] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:22.081423 6616 raft_consensus.cc:482] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:22.081444 6616 raft_consensus.cc:3037] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:22.082309 6616 raft_consensus.cc:504] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.082456 6616 leader_election.cc:288] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 election: Requested vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979)
I20260807 06:33:22.082643 6044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05b9aa1d31459699b4f4e7e36ac442" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f"
I20260807 06:33:22.082654 6250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05b9aa1d31459699b4f4e7e36ac442" candidate_uuid: "3d2e2d5c73174969b33aa85c6430f25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a258348e35404f75ad925d9b5c9f23a0"
I20260807 06:33:22.082710 6250 raft_consensus.cc:3037] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:22.082710 6044 raft_consensus.cc:3037] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:22.083473 6250 raft_consensus.cc:2454] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 1.
I20260807 06:33:22.083498 6044 raft_consensus.cc:2454] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d2e2d5c73174969b33aa85c6430f25f in term 1.
I20260807 06:33:22.083637 6364 leader_election.cc:302] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, a258348e35404f75ad925d9b5c9f23a0; no voters:
I20260807 06:33:22.083739 6616 raft_consensus.cc:2781] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:22.083788 6616 raft_consensus.cc:686] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 LEADER]: Becoming Leader. State: Replica: 3d2e2d5c73174969b33aa85c6430f25f, State: Running, Role: LEADER
I20260807 06:33:22.083843 6616 consensus_queue.cc:237] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:22.084323 5794 catalog_manager.cc:5495] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3). New cstate: current_term: 1 leader_uuid: "3d2e2d5c73174969b33aa85c6430f25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } health_report { overall_health: HEALTHY } } }
I20260807 06:33:22.095216 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.092821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.2.243.3:45527)
0807 06:33:22.092950 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:22.092952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:22.092958 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:33:22.092974 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:22.092975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:22.092982 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:22.093045 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:22.093186 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.093740 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.094493 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.094745 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.094805 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.094947 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.094979 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.095042 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:22.095111 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 06:33:22.095129 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 06:33:22.099324 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.097288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:60791 (local address 127.2.243.3:45527)
0807 06:33:22.097337 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:22.097339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:22.097364 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:33:22.097444 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:22.097445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:22.097453 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:22.097499 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:22.097596 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.098003 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.098707 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.098892 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.098966 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.099101 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.099128 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.099178 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:22.099233 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 06:33:22.099247 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:33:22.099327 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:22.097258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45527 (local address 127.2.243.3:60791)
0807 06:33:22.097304 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:22.097314 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:22.097378 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:22.097501 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:22.097502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:22.097503 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:22.097534 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.097535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.098076 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.098077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.098619 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:22.098620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:22.098966 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:22.098967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:22.099017 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:22.099023 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:22.099180 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:22.099182 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:22.099230 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 06:33:22.099265 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 06:33:22.101574 6044 raft_consensus.cc:1261] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Refusing update from remote peer 3d2e2d5c73174969b33aa85c6430f25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:22.101598 6250 raft_consensus.cc:1261] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Refusing update from remote peer 3d2e2d5c73174969b33aa85c6430f25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:22.101780 6616 consensus_queue.cc:1035] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:22.101867 6641 consensus_queue.cc:1035] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:33:22.107122 6621 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 06:33:23.630119 5794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:23.630234 5794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:23.631067 5794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Sending DeleteTablet for 3 replicas of tablet 3c05b9aa1d31459699b4f4e7e36ac442
I20260807 06:33:23.631331 6009 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c05b9aa1d31459699b4f4e7e36ac442 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:23 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:33284
I20260807 06:33:23.631434 6216 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c05b9aa1d31459699b4f4e7e36ac442 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:23 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:48242
I20260807 06:33:23.631608 6426 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c05b9aa1d31459699b4f4e7e36ac442 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:23 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:57908
I20260807 06:33:23.631810 6702 tablet_replica.cc:330] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: stopping tablet replica
I20260807 06:33:23.631812 6700 tablet_replica.cc:330] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: stopping tablet replica
I20260807 06:33:23.631875 6700 raft_consensus.cc:2229] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:23.631901 6702 raft_consensus.cc:2229] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:23.631960 6700 raft_consensus.cc:2258] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:23.631985 6702 raft_consensus.cc:2258] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:23.632230 6702 ts_tablet_manager.cc:1905] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:23.632247 6700 ts_tablet_manager.cc:1905] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:23.634127 6702 ts_tablet_manager.cc:1918] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:33:23.634205 6702 log.cc:1196] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/3c05b9aa1d31459699b4f4e7e36ac442
I20260807 06:33:23.634428 6700 ts_tablet_manager.cc:1918] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:33:23.634466 6702 ts_tablet_manager.cc:1939] T 3c05b9aa1d31459699b4f4e7e36ac442 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting consensus metadata
I20260807 06:33:23.634506 6700 log.cc:1196] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/3c05b9aa1d31459699b4f4e7e36ac442
I20260807 06:33:23.634748 6700 ts_tablet_manager.cc:1939] T 3c05b9aa1d31459699b4f4e7e36ac442 P 4942c6bb37474baf8de29784c396644f: Deleting consensus metadata
I20260807 06:33:23.634824 6701 tablet_replica.cc:330] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: stopping tablet replica
I20260807 06:33:23.634898 6701 raft_consensus.cc:2229] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:23.634918 5776 catalog_manager.cc:4841] TS 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527): tablet 3c05b9aa1d31459699b4f4e7e36ac442 (table test-table [id=515e41eb6d2649818ff29870333c477a]) successfully deleted
I20260807 06:33:23.634953 6701 raft_consensus.cc:2258] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:23.635198 5771 catalog_manager.cc:4841] TS 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019): tablet 3c05b9aa1d31459699b4f4e7e36ac442 (table test-table [id=515e41eb6d2649818ff29870333c477a]) successfully deleted
I20260807 06:33:23.635255 6701 ts_tablet_manager.cc:1905] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:23.637094 6701 ts_tablet_manager.cc:1918] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:33:23.637162 6701 log.cc:1196] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/3c05b9aa1d31459699b4f4e7e36ac442
I20260807 06:33:23.637374 6701 ts_tablet_manager.cc:1939] T 3c05b9aa1d31459699b4f4e7e36ac442 P a258348e35404f75ad925d9b5c9f23a0: Deleting consensus metadata
I20260807 06:33:23.637818 5774 catalog_manager.cc:4841] TS a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979): tablet 3c05b9aa1d31459699b4f4e7e36ac442 (table test-table [id=515e41eb6d2649818ff29870333c477a]) successfully deleted
Aug 07 06:33:23 dist-test-slave-rwrg krb5kdc[5734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:33:23 dist-test-slave-rwrg krb5kdc[5734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:23.659010 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.650858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.2.243.62:46547)
0807 06:33:23.651018 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:23.651020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:23.651082 (+ 62us) server_negotiation.cc:407] Connection header received
0807 06:33:23.651159 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:23.651160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:23.651170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:23.651238 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:23.651962 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.652587 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.653492 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.653695 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.653763 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.656049 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:23.656050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:23.656051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:23.656058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:23.657672 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:23.658042 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:23.658043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:23.658044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:23.658101 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:23.658339 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:23.658341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:23.658341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:23.658465 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:23.658577 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:23.658853 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 06:33:23.658927 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":102,"threads_started":1}
I20260807 06:33:23.660576 5794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:23.660750 5794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:23.662230 6009 tablet_service.cc:1467] Processing CreateTablet for tablet b4f2ba3dc0df4d9daa280ec52208b335 (DEFAULT_TABLE table=test-table [id=d63761b461da45ca98854ab228d88248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:23.662279 6216 tablet_service.cc:1467] Processing CreateTablet for tablet b4f2ba3dc0df4d9daa280ec52208b335 (DEFAULT_TABLE table=test-table [id=d63761b461da45ca98854ab228d88248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:23.662352 6426 tablet_service.cc:1467] Processing CreateTablet for tablet b4f2ba3dc0df4d9daa280ec52208b335 (DEFAULT_TABLE table=test-table [id=d63761b461da45ca98854ab228d88248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:23.662418 6216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f2ba3dc0df4d9daa280ec52208b335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:23.662418 6009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f2ba3dc0df4d9daa280ec52208b335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:23.662509 6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f2ba3dc0df4d9daa280ec52208b335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:23.663973 6723 tablet_bootstrap.cc:492] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: Bootstrap starting.
I20260807 06:33:23.664371 6723 tablet_bootstrap.cc:654] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:23.664593 6725 tablet_bootstrap.cc:492] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: Bootstrap starting.
I20260807 06:33:23.664999 6725 tablet_bootstrap.cc:654] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:23.665053 6723 tablet_bootstrap.cc:492] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: No bootstrap required, opened a new log
I20260807 06:33:23.665099 6723 ts_tablet_manager.cc:1397] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:23.665328 6724 tablet_bootstrap.cc:492] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: Bootstrap starting.
I20260807 06:33:23.665323 6723 raft_consensus.cc:348] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.665563 6723 raft_consensus.cc:374] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:23.665635 6723 raft_consensus.cc:729] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d2e2d5c73174969b33aa85c6430f25f, State: Initialized, Role: FOLLOWER
I20260807 06:33:23.665714 6725 tablet_bootstrap.cc:492] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: No bootstrap required, opened a new log
I20260807 06:33:23.665709 6723 consensus_queue.cc:260] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.665768 6725 ts_tablet_manager.cc:1397] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:23.665771 6724 tablet_bootstrap.cc:654] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:23.665936 6723 ts_tablet_manager.cc:1428] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:23.666029 6725 raft_consensus.cc:348] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.666123 6725 raft_consensus.cc:374] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:23.666158 6725 raft_consensus.cc:729] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a258348e35404f75ad925d9b5c9f23a0, State: Initialized, Role: FOLLOWER
I20260807 06:33:23.666213 6725 consensus_queue.cc:260] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.666385 6725 ts_tablet_manager.cc:1428] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:23.666872 6724 tablet_bootstrap.cc:492] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: No bootstrap required, opened a new log
I20260807 06:33:23.666961 6724 ts_tablet_manager.cc:1397] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:23.667135 6724 raft_consensus.cc:348] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.667219 6724 raft_consensus.cc:374] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:23.667253 6724 raft_consensus.cc:729] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4942c6bb37474baf8de29784c396644f, State: Initialized, Role: FOLLOWER
I20260807 06:33:23.667302 6724 consensus_queue.cc:260] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.667507 6724 ts_tablet_manager.cc:1428] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:23.768735 6729 raft_consensus.cc:482] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:23.768857 6729 raft_consensus.cc:504] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.769222 6729 leader_election.cc:288] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527)
I20260807 06:33:23.772543 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.769469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:59263 (local address 127.2.243.1:41019)
0807 06:33:23.769684 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:23.769687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:23.769861 (+ 174us) server_negotiation.cc:407] Connection header received
0807 06:33:23.769964 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:23.769965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:23.769976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:23.770063 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:23.770224 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.770772 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.771733 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.772142 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.772229 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.772240 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:23.772389 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 06:33:23.772432 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":115,"threads_started":1}
I20260807 06:33:23.772540 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.769386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:41019 (local address 127.2.243.2:59263)
0807 06:33:23.769780 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:23.769791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:23.769878 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:23.770067 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:23.770067 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:23.770070 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:23.770150 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.770152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.770866 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.770867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.771634 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.771637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.772231 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.772233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.772312 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.772315 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:23.772382 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:33:23.772421 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":341,"threads_started":1}
I20260807 06:33:23.772779 6043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f2ba3dc0df4d9daa280ec52208b335" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f" is_pre_election: true
I20260807 06:33:23.772861 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.769719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45527 (local address 127.2.243.2:51425)
0807 06:33:23.769864 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:23.769874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:23.769959 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:23.770138 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:23.770139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:23.770141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:23.770613 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.770616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.771277 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.771278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.771998 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.772000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.772591 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.772592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.772675 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.772678 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:23.772737 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:33:23.772773 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":102,"threads_started":1}
I20260807 06:33:23.772869 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.769497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:51425 (local address 127.2.243.3:45527)
0807 06:33:23.769729 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:23.769731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:23.769950 (+ 219us) server_negotiation.cc:407] Connection header received
0807 06:33:23.770025 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:23.770026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:23.770037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:23.770122 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:23.770718 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.771193 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.772091 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.772480 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.772589 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.772601 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:23.772751 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 06:33:23.772805 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":125,"threads_started":1}
I20260807 06:33:23.772920 6043 raft_consensus.cc:2454] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 0.
I20260807 06:33:23.773149 6148 leader_election.cc:302] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4942c6bb37474baf8de29784c396644f, a258348e35404f75ad925d9b5c9f23a0; no voters:
I20260807 06:33:23.773149 6453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f2ba3dc0df4d9daa280ec52208b335" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d2e2d5c73174969b33aa85c6430f25f" is_pre_election: true
I20260807 06:33:23.773257 6453 raft_consensus.cc:2454] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 0.
I20260807 06:33:23.773276 6729 raft_consensus.cc:2781] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:23.773303 6729 raft_consensus.cc:482] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:23.773324 6729 raft_consensus.cc:3037] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:23.774070 6729 raft_consensus.cc:504] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.774224 6729 leader_election.cc:288] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 election: Requested vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527)
I20260807 06:33:23.774348 6043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f2ba3dc0df4d9daa280ec52208b335" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f"
I20260807 06:33:23.774376 6453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4f2ba3dc0df4d9daa280ec52208b335" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d2e2d5c73174969b33aa85c6430f25f"
I20260807 06:33:23.774433 6453 raft_consensus.cc:3037] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:23.774434 6043 raft_consensus.cc:3037] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:23.775262 6043 raft_consensus.cc:2454] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 1.
I20260807 06:33:23.775265 6453 raft_consensus.cc:2454] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 1.
I20260807 06:33:23.775588 6153 leader_election.cc:302] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, a258348e35404f75ad925d9b5c9f23a0; no voters:
I20260807 06:33:23.775712 6729 raft_consensus.cc:2781] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:23.775944 6729 raft_consensus.cc:686] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 1 LEADER]: Becoming Leader. State: Replica: a258348e35404f75ad925d9b5c9f23a0, State: Running, Role: LEADER
I20260807 06:33:23.776028 6729 consensus_queue.cc:237] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.776847 5794 catalog_manager.cc:5495] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2). New cstate: current_term: 1 leader_uuid: "a258348e35404f75ad925d9b5c9f23a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } health_report { overall_health: UNKNOWN } } }
I20260807 06:33:23.813165 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.810467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.2.243.2:40979)
0807 06:33:23.810640 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:23.810641 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:23.810648 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:33:23.810682 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:23.810683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:23.810703 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:23.810799 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:23.810950 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.811469 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.812247 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.812475 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.812571 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.812777 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:23.812896 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:23.812988 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:23.813044 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 06:33:23.813066 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260807 06:33:23.818220 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.815823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:55663 (local address 127.2.243.3:45527)
0807 06:33:23.815866 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:23.815868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:23.815876 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:23.815940 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:23.815941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:23.815954 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:23.816016 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:23.816175 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.816641 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.817455 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.817679 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.817759 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.817949 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:23.818000 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:23.818061 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:23.818122 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 06:33:23.818144 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:23.818259 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.815638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45527 (local address 127.2.243.2:55663)
0807 06:33:23.815814 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:23.815825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:23.815880 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:23.816047 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:23.816047 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:23.816049 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:23.816106 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.816112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.816738 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.816739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.817367 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.817369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.817759 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.817760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.817832 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.817853 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:23.818064 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:23.818066 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:23.818118 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:33:23.818159 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260807 06:33:23.820667 6043 raft_consensus.cc:1261] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Refusing update from remote peer a258348e35404f75ad925d9b5c9f23a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:23.820662 6453 raft_consensus.cc:1261] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Refusing update from remote peer a258348e35404f75ad925d9b5c9f23a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:23.820968 6729 consensus_queue.cc:1035] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:23.821148 6737 consensus_queue.cc:1035] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:23.828338 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.826006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:44585 (local address 127.2.243.2:40979)
0807 06:33:23.826064 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:23.826066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:23.826237 (+ 171us) server_negotiation.cc:407] Connection header received
0807 06:33:23.826279 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:23.826279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:23.826290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:23.826340 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:23.826491 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.826952 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.827651 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.827865 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.827929 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.828094 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:23.828129 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:23.828175 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:23.828244 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 06:33:23.828267 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:33:23.828370 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:23.825966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:40979 (local address 127.2.243.3:44585)
0807 06:33:23.826114 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:23.826126 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:23.826180 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:23.826361 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:23.826362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:23.826365 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:23.826416 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.826418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.827018 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.827018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.827577 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:23.827578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:23.827939 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:23.827939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:23.827999 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:23.828005 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:23.828190 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:23.828192 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:23.828232 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 06:33:23.828285 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260807 06:33:23.835161 5793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:23.835280 5793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:23.835908 5793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Sending DeleteTablet for 3 replicas of tablet b4f2ba3dc0df4d9daa280ec52208b335
I20260807 06:33:23.836139 6009 tablet_service.cc:1514] Processing DeleteTablet for tablet b4f2ba3dc0df4d9daa280ec52208b335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:23 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:33284
I20260807 06:33:23.836169 6216 tablet_service.cc:1514] Processing DeleteTablet for tablet b4f2ba3dc0df4d9daa280ec52208b335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:23 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:48242
I20260807 06:33:23.836190 6426 tablet_service.cc:1514] Processing DeleteTablet for tablet b4f2ba3dc0df4d9daa280ec52208b335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:23 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:57908
I20260807 06:33:23.836280 6700 tablet_replica.cc:330] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: stopping tablet replica
I20260807 06:33:23.836282 6701 tablet_replica.cc:330] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: stopping tablet replica
I20260807 06:33:23.836287 6702 tablet_replica.cc:330] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: stopping tablet replica
I20260807 06:33:23.836333 6701 raft_consensus.cc:2229] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:23.836340 6700 raft_consensus.cc:2229] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:23.836342 6702 raft_consensus.cc:2229] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:23.836377 6702 raft_consensus.cc:2258] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:23.836380 6700 raft_consensus.cc:2258] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:23.836417 6701 raft_consensus.cc:2258] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:23.836742 6702 ts_tablet_manager.cc:1905] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:23.836777 6701 ts_tablet_manager.cc:1905] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:23.836796 6700 ts_tablet_manager.cc:1905] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:23.838347 6700 ts_tablet_manager.cc:1918] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:23.838403 6700 log.cc:1196] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/b4f2ba3dc0df4d9daa280ec52208b335
I20260807 06:33:23.838397 6701 ts_tablet_manager.cc:1918] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:23.838451 6701 log.cc:1196] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/b4f2ba3dc0df4d9daa280ec52208b335
I20260807 06:33:23.838640 6700 ts_tablet_manager.cc:1939] T b4f2ba3dc0df4d9daa280ec52208b335 P 4942c6bb37474baf8de29784c396644f: Deleting consensus metadata
I20260807 06:33:23.838680 6701 ts_tablet_manager.cc:1939] T b4f2ba3dc0df4d9daa280ec52208b335 P a258348e35404f75ad925d9b5c9f23a0: Deleting consensus metadata
I20260807 06:33:23.838698 5793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:23.838860 5793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:23.838979 5774 catalog_manager.cc:4841] TS a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979): tablet b4f2ba3dc0df4d9daa280ec52208b335 (table test-table [id=d63761b461da45ca98854ab228d88248]) successfully deleted
I20260807 06:33:23.839085 5771 catalog_manager.cc:4841] TS 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019): tablet b4f2ba3dc0df4d9daa280ec52208b335 (table test-table [id=d63761b461da45ca98854ab228d88248]) successfully deleted
I20260807 06:33:23.839745 6702 ts_tablet_manager.cc:1918] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:23.839803 6702 log.cc:1196] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/b4f2ba3dc0df4d9daa280ec52208b335
I20260807 06:33:23.840034 6702 ts_tablet_manager.cc:1939] T b4f2ba3dc0df4d9daa280ec52208b335 P 3d2e2d5c73174969b33aa85c6430f25f: Deleting consensus metadata
I20260807 06:33:23.840075 6009 tablet_service.cc:1467] Processing CreateTablet for tablet 38b52ce3170043478848c25547347e6f (DEFAULT_TABLE table=test-table [id=e71bfd08864648479410ec7ad61ab00f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:23.840170 6009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b52ce3170043478848c25547347e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:23.840708 5776 catalog_manager.cc:4841] TS 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527): tablet b4f2ba3dc0df4d9daa280ec52208b335 (table test-table [id=d63761b461da45ca98854ab228d88248]) successfully deleted
I20260807 06:33:23.840775 6426 tablet_service.cc:1467] Processing CreateTablet for tablet 38b52ce3170043478848c25547347e6f (DEFAULT_TABLE table=test-table [id=e71bfd08864648479410ec7ad61ab00f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:23.840853 6216 tablet_service.cc:1467] Processing CreateTablet for tablet 38b52ce3170043478848c25547347e6f (DEFAULT_TABLE table=test-table [id=e71bfd08864648479410ec7ad61ab00f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:23.840870 6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b52ce3170043478848c25547347e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:23.840962 6216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b52ce3170043478848c25547347e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:23.841394 6724 tablet_bootstrap.cc:492] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f: Bootstrap starting.
I20260807 06:33:23.841702 6724 tablet_bootstrap.cc:654] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:23.842156 6725 tablet_bootstrap.cc:492] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: Bootstrap starting.
I20260807 06:33:23.842157 6723 tablet_bootstrap.cc:492] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: Bootstrap starting.
I20260807 06:33:23.842429 6723 tablet_bootstrap.cc:654] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:23.842581 6725 tablet_bootstrap.cc:654] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:23.842981 6723 tablet_bootstrap.cc:492] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: No bootstrap required, opened a new log
I20260807 06:33:23.843037 6723 ts_tablet_manager.cc:1397] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:23.843137 6723 raft_consensus.cc:348] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.843182 6723 raft_consensus.cc:374] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:23.843195 6723 raft_consensus.cc:729] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d2e2d5c73174969b33aa85c6430f25f, State: Initialized, Role: FOLLOWER
I20260807 06:33:23.843211 6725 tablet_bootstrap.cc:492] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: No bootstrap required, opened a new log
I20260807 06:33:23.843254 6725 ts_tablet_manager.cc:1397] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:23.843261 6723 consensus_queue.cc:260] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.843358 6723 ts_tablet_manager.cc:1428] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:23.843416 6725 raft_consensus.cc:348] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.843475 6725 raft_consensus.cc:374] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:23.843497 6725 raft_consensus.cc:729] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a258348e35404f75ad925d9b5c9f23a0, State: Initialized, Role: FOLLOWER
I20260807 06:33:23.843537 6725 consensus_queue.cc:260] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.843648 6725 ts_tablet_manager.cc:1428] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:23.844260 6724 tablet_bootstrap.cc:492] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f: No bootstrap required, opened a new log
I20260807 06:33:23.844304 6724 ts_tablet_manager.cc:1397] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:33:23.844409 6724 raft_consensus.cc:348] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.844473 6724 raft_consensus.cc:374] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:23.844527 6724 raft_consensus.cc:729] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4942c6bb37474baf8de29784c396644f, State: Initialized, Role: FOLLOWER
I20260807 06:33:23.844581 6724 consensus_queue.cc:260] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:23.844679 6724 ts_tablet_manager.cc:1428] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:24.083340 6753 raft_consensus.cc:482] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:24.083448 6753 raft_consensus.cc:504] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:24.083634 6753 leader_election.cc:288] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527)
I20260807 06:33:24.083866 6043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b52ce3170043478848c25547347e6f" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f" is_pre_election: true
I20260807 06:33:24.083961 6043 raft_consensus.cc:2454] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 0.
I20260807 06:33:24.083922 6453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b52ce3170043478848c25547347e6f" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d2e2d5c73174969b33aa85c6430f25f" is_pre_election: true
I20260807 06:33:24.084067 6453 raft_consensus.cc:2454] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 0.
I20260807 06:33:24.084161 6148 leader_election.cc:302] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4942c6bb37474baf8de29784c396644f, a258348e35404f75ad925d9b5c9f23a0; no voters:
I20260807 06:33:24.084287 6753 raft_consensus.cc:2781] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:24.084358 6753 raft_consensus.cc:482] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:24.084384 6753 raft_consensus.cc:3037] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:24.085151 6753 raft_consensus.cc:504] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:24.085301 6753 leader_election.cc:288] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 election: Requested vote from peers 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019), 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527)
I20260807 06:33:24.085451 6453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b52ce3170043478848c25547347e6f" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d2e2d5c73174969b33aa85c6430f25f"
I20260807 06:33:24.085520 6453 raft_consensus.cc:3037] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:24.085458 6043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b52ce3170043478848c25547347e6f" candidate_uuid: "a258348e35404f75ad925d9b5c9f23a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4942c6bb37474baf8de29784c396644f"
I20260807 06:33:24.085588 6043 raft_consensus.cc:3037] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:24.086198 6453 raft_consensus.cc:2454] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 1.
I20260807 06:33:24.086205 6043 raft_consensus.cc:2454] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a258348e35404f75ad925d9b5c9f23a0 in term 1.
I20260807 06:33:24.086371 6153 leader_election.cc:302] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d2e2d5c73174969b33aa85c6430f25f, a258348e35404f75ad925d9b5c9f23a0; no voters:
I20260807 06:33:24.086480 6753 raft_consensus.cc:2781] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:24.086572 6753 raft_consensus.cc:686] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 1 LEADER]: Becoming Leader. State: Replica: a258348e35404f75ad925d9b5c9f23a0, State: Running, Role: LEADER
I20260807 06:33:24.086627 6753 consensus_queue.cc:237] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } }
I20260807 06:33:24.087220 5793 catalog_manager.cc:5495] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2). New cstate: current_term: 1 leader_uuid: "a258348e35404f75ad925d9b5c9f23a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a258348e35404f75ad925d9b5c9f23a0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 } health_report { overall_health: UNKNOWN } } }
I20260807 06:33:24.130162 6453 raft_consensus.cc:1261] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Refusing update from remote peer a258348e35404f75ad925d9b5c9f23a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:24.130195 6043 raft_consensus.cc:1261] T 38b52ce3170043478848c25547347e6f P 4942c6bb37474baf8de29784c396644f [term 1 FOLLOWER]: Refusing update from remote peer a258348e35404f75ad925d9b5c9f23a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:24.130398 6753 consensus_queue.cc:1035] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4942c6bb37474baf8de29784c396644f" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:24.130481 6729 consensus_queue.cc:1035] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d2e2d5c73174969b33aa85c6430f25f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:33:24.135023 6679 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 06:33:25.653522 5792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:25.653659 5792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:25.654805 5792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa8dd5c8f41c446db21eebaf72300d66: Sending DeleteTablet for 3 replicas of tablet 38b52ce3170043478848c25547347e6f
I20260807 06:33:25.655154 6426 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b52ce3170043478848c25547347e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:25 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:57908
I20260807 06:33:25.655213 6216 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b52ce3170043478848c25547347e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:25 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:48242
I20260807 06:33:25.655332 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5906
I20260807 06:33:25.655409 6009 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b52ce3170043478848c25547347e6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:25 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:33284
I20260807 06:33:25.663295 6802 tablet_replica.cc:330] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: stopping tablet replica
I20260807 06:33:25.663400 6802 raft_consensus.cc:2229] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:25.663489 6802 raft_consensus.cc:2258] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:25.665859 6802 ts_tablet_manager.cc:1905] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:25.668304 6802 ts_tablet_manager.cc:1918] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:33:25.668395 6802 log.cc:1196] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/38b52ce3170043478848c25547347e6f
I20260807 06:33:25.668641 6801 tablet_replica.cc:330] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: stopping tablet replica
I20260807 06:33:25.668735 6801 raft_consensus.cc:2229] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:25.668751 6802 ts_tablet_manager.cc:1939] T 38b52ce3170043478848c25547347e6f P a258348e35404f75ad925d9b5c9f23a0: Deleting consensus metadata
I20260807 06:33:25.668802 6801 raft_consensus.cc:2258] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:25.669163 5774 catalog_manager.cc:4841] TS a258348e35404f75ad925d9b5c9f23a0 (127.2.243.2:40979): tablet 38b52ce3170043478848c25547347e6f (table test-table [id=e71bfd08864648479410ec7ad61ab00f]) successfully deleted
I20260807 06:33:25.669230 6801 ts_tablet_manager.cc:1905] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:25.670722 6801 ts_tablet_manager.cc:1918] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:33:25.670782 6801 log.cc:1196] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/38b52ce3170043478848c25547347e6f
I20260807 06:33:25.671049 6801 ts_tablet_manager.cc:1939] T 38b52ce3170043478848c25547347e6f P 3d2e2d5c73174969b33aa85c6430f25f: Deleting consensus metadata
I20260807 06:33:25.671471 5776 catalog_manager.cc:4841] TS 3d2e2d5c73174969b33aa85c6430f25f (127.2.243.3:45527): tablet 38b52ce3170043478848c25547347e6f (table test-table [id=e71bfd08864648479410ec7ad61ab00f]) successfully deleted
W20260807 06:33:25.678184 5771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:41019 (error 108)
I20260807 06:33:25.678856 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6117
I20260807 06:33:25.678927 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:25.678678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:41019 (local address 127.0.0.1:60412)
0807 06:33:25.678820 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:25.678874 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:41019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
W20260807 06:33:25.687485 5771 catalog_manager.cc:4584] TS 4942c6bb37474baf8de29784c396644f (127.2.243.1:41019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38b52ce3170043478848c25547347e6f: Network error: Client connection negotiation failed: client connection to 127.2.243.1:41019: connect: Connection refused (error 111)
I20260807 06:33:25.692750 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6335
I20260807 06:33:25.706027 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5750
2026-08-07T06:33:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6272 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:25 dist-test-slave-rwrg krb5kdc[6812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:25 dist-test-slave-rwrg krb5kdc[6812](info): set up 2 sockets
Aug 07 06:33:25 dist-test-slave-rwrg krb5kdc[6812](info): commencing operation
krb5kdc: starting...
W20260807 06:33:26.971475 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:27.095018 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43095
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:43095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:27.177142 6836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:27.177397 6836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:27.177433 6836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:27.178565 6836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:27.178642 6836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:27.178671 6836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:27.178702 6836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:27.178727 6836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:27.180200 6836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:43095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:27.180472 6836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:27.180730 6836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:27.185851 6842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:27.185887 6843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:27.185887 6846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:27.185928 6836 server_base.cc:1034] running on GCE node
I20260807 06:33:27.186328 6836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:27.186606 6836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:27.187788 6836 hybrid_clock.cc:648] HybridClock initialized: now 1786084407187773 us; error 53 us; skew 500 ppm
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:27.193507 6836 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:27.193991 6836 webserver.cc:460] Webserver started at http://127.2.243.62:40471/ using document root <none> and password file <none>
I20260807 06:33:27.194247 6836 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:27.194320 6836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:27.194475 6836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:27.195461 6836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "a711dd76762a496f896775ed7a20c081"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "b09a97d02bedeb491cec424562ec78cb"
server_key_iv: "f3727bba588e4af6b064c0494104cd18"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.195811 6836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "a711dd76762a496f896775ed7a20c081"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "b09a97d02bedeb491cec424562ec78cb"
server_key_iv: "f3727bba588e4af6b064c0494104cd18"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.197168 6836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:27.197860 6855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.198064 6836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:27.198179 6836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "a711dd76762a496f896775ed7a20c081"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "b09a97d02bedeb491cec424562ec78cb"
server_key_iv: "f3727bba588e4af6b064c0494104cd18"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.198251 6836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:27.215453 6836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:27.217512 6836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:27.217689 6836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:27.221982 6836 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:43095
I20260807 06:33:27.222025 6944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:43095 every 8 connection(s)
I20260807 06:33:27.222405 6836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:27.223259 6945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:27.225627 6945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081: Bootstrap starting.
I20260807 06:33:27.226264 6945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:27.226576 6945 log.cc:824] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:27.227278 6945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081: No bootstrap required, opened a new log
I20260807 06:33:27.228988 6945 raft_consensus.cc:348] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a711dd76762a496f896775ed7a20c081" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43095 } }
I20260807 06:33:27.229122 6945 raft_consensus.cc:374] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:27.229146 6945 raft_consensus.cc:729] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a711dd76762a496f896775ed7a20c081, State: Initialized, Role: FOLLOWER
I20260807 06:33:27.229319 6945 consensus_queue.cc:260] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a711dd76762a496f896775ed7a20c081" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43095 } }
I20260807 06:33:27.229384 6945 raft_consensus.cc:388] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:27.229437 6945 raft_consensus.cc:482] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:27.229489 6945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:27.230024 6945 raft_consensus.cc:504] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a711dd76762a496f896775ed7a20c081" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43095 } }
I20260807 06:33:27.230147 6945 leader_election.cc:302] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a711dd76762a496f896775ed7a20c081; no voters:
I20260807 06:33:27.230274 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6836
I20260807 06:33:27.230324 6945 leader_election.cc:288] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:27.230370 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:27.230400 6950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:27.230568 3020 external_mini_cluster.cc:1428] Setting key 9ab0bdfa01c7c16336c6686f48c652e1
I20260807 06:33:27.230595 6945 sys_catalog.cc:564] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:27.230841 6950 raft_consensus.cc:686] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [term 1 LEADER]: Becoming Leader. State: Replica: a711dd76762a496f896775ed7a20c081, State: Running, Role: LEADER
I20260807 06:33:27.230996 6950 consensus_queue.cc:237] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a711dd76762a496f896775ed7a20c081" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43095 } }
I20260807 06:33:27.231644 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a711dd76762a496f896775ed7a20c081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a711dd76762a496f896775ed7a20c081" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43095 } } }
I20260807 06:33:27.231766 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:27.231644 6953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a711dd76762a496f896775ed7a20c081. Latest consensus state: current_term: 1 leader_uuid: "a711dd76762a496f896775ed7a20c081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a711dd76762a496f896775ed7a20c081" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43095 } } }
I20260807 06:33:27.232095 6953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:27.232342 6973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:27.233043 6973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:27.237025 6973 catalog_manager.cc:1347] Generated new cluster ID: 841884be5d0a47c2b9455ff0d0d43419
I20260807 06:33:27.237085 6973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:27.242045 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:27.231265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.2.243.62:43095)
0807 06:33:27.231414 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:27.231422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:27.231462 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:33:27.231585 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:27.231592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:27.231657 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:27.232682 (+ 1025us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:27.232762 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.233787 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.234836 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.235250 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.235334 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:27.237953 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:27.237957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:27.237957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:27.237969 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:27.240359 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.240781 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.240785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.240785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.240875 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.241141 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.241142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.241143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.241375 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:27.241537 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:27.241655 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 06:33:27.241883 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":80,"threads_started":1}
I20260807 06:33:27.264215 6973 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:27.264775 6973 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:27.273706 6973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a711dd76762a496f896775ed7a20c081: Generated new TSK 0
I20260807 06:33:27.273906 6973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:27.307740 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43095
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:27.390553 6984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:27.390774 6984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:27.390796 6984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:27.391880 6984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:27.391914 6984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:27.391939 6984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:27.393469 6984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:27.393682 6984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:27.393847 6984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:27.398801 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:27.398815 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:27.398823 6997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:27.399174 6984 server_base.cc:1034] running on GCE node
I20260807 06:33:27.399369 6984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:27.399624 6984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:27.400792 6984 hybrid_clock.cc:648] HybridClock initialized: now 1786084407400772 us; error 30 us; skew 500 ppm
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:27.403534 6984 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:27.403993 6984 webserver.cc:460] Webserver started at http://127.2.243.1:45167/ using document root <none> and password file <none>
I20260807 06:33:27.404187 6984 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:27.404229 6984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:27.404328 6984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:27.405378 6984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "8cbf1cd110844058b6e76bd80df93939"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "65a555b309f8fa144a568b8e1e918412"
server_key_iv: "8273e4f7978e8f09ebb6876ee97903ce"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.405702 6984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "8cbf1cd110844058b6e76bd80df93939"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "65a555b309f8fa144a568b8e1e918412"
server_key_iv: "8273e4f7978e8f09ebb6876ee97903ce"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.406965 6984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:33:27.407637 7008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.407799 6984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:27.407886 6984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "8cbf1cd110844058b6e76bd80df93939"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "65a555b309f8fa144a568b8e1e918412"
server_key_iv: "8273e4f7978e8f09ebb6876ee97903ce"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.407939 6984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:27.416035 6984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:27.417910 6984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:27.418026 6984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:27.418206 6984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:27.418581 6984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:27.418665 6984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.418706 6984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:27.418727 6984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.424448 6984 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40785
I20260807 06:33:27.424553 7192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40785 every 8 connection(s)
I20260807 06:33:27.424886 6984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:27.433022 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6984
I20260807 06:33:27.433108 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:27.433358 3020 external_mini_cluster.cc:1428] Setting key 4f8f7f9923d2d03e607ca1a434bbae38
I20260807 06:33:27.434793 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:27.425714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:41863 (local address 127.2.243.62:43095)
0807 06:33:27.425779 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:27.425781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:27.425963 (+ 182us) server_negotiation.cc:407] Connection header received
0807 06:33:27.426481 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:27.426483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:27.426499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:27.426559 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:27.427502 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.428035 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.429095 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.429315 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.429394 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:27.431604 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:27.431605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:27.431606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:27.431613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:27.433300 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.433793 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.433794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.433798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.433847 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.434128 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.434129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.434130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.434248 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:27.434318 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:27.434612 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 06:33:27.434700 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:33:27.435000 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:27.425647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43095 (local address 127.2.243.1:41863)
0807 06:33:27.425825 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:27.425875 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:27.426371 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:27.426636 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:27.426640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:27.427214 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:27.427397 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:27.427401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.428131 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.428132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:27.429001 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:27.429004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.429403 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.429404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:27.429532 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:27.430064 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:27.430073 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:27.431503 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:27.433378 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:27.433385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:27.433387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:27.433645 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:27.433923 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:27.433923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:27.433924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:27.434058 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:27.434321 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:27.434328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:27.434464 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 06:33:27.434590 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 06:33:27.434757 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260807 06:33:27.435748 7193 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43095
I20260807 06:33:27.435865 7193 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:27.436061 7193 heartbeater.cc:511] Master 127.2.243.62:43095 requested a full tablet report, sending...
I20260807 06:33:27.436760 6881 ts_manager.cc:194] Registered new tserver with Master: 8cbf1cd110844058b6e76bd80df93939 (127.2.243.1:40785)
I20260807 06:33:27.437487 6881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:41863
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:27.464207 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43095
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:27.547852 7201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:27.548079 7201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:27.548103 7201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:27.549809 7201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:27.549858 7201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:27.549885 7201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:27.551353 7201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:27.551612 7201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:27.551836 7201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:27.556739 7212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:27.556767 7209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:27.556743 7210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:27.557098 7201 server_base.cc:1034] running on GCE node
I20260807 06:33:27.557408 7201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:27.557693 7201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:27.558857 7201 hybrid_clock.cc:648] HybridClock initialized: now 1786084407558837 us; error 31 us; skew 500 ppm
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:27.561496 7201 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:27.562009 7201 webserver.cc:460] Webserver started at http://127.2.243.2:43237/ using document root <none> and password file <none>
I20260807 06:33:27.562261 7201 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:27.562330 7201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:27.562487 7201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:27.563575 7201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "d4224727be0a495c80e321cd0f2834ca"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "3da2903853fc938767d87e34f39ca331"
server_key_iv: "1431e305e8a47359a49d7d8e3b17d653"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.563948 7201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "d4224727be0a495c80e321cd0f2834ca"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "3da2903853fc938767d87e34f39ca331"
server_key_iv: "1431e305e8a47359a49d7d8e3b17d653"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.565377 7201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:27.566170 7227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.566344 7201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:27.566447 7201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "d4224727be0a495c80e321cd0f2834ca"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "3da2903853fc938767d87e34f39ca331"
server_key_iv: "1431e305e8a47359a49d7d8e3b17d653"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.566563 7201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:27.578457 7201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:27.580775 7201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:27.580940 7201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:27.581176 7201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:27.581475 7201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:27.581537 7201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.581578 7201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:27.581605 7201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.587630 7201 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:33707
I20260807 06:33:27.587683 7417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:33707 every 8 connection(s)
I20260807 06:33:27.588050 7201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:27.589504 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 7201
I20260807 06:33:27.589608 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:27.589818 3020 external_mini_cluster.cc:1428] Setting key 1788ba1279d6b9ad4df2541ed9b6891b
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:27.598680 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:27.588899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:35159 (local address 127.2.243.62:43095)
0807 06:33:27.588988 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:27.588990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:27.589184 (+ 194us) server_negotiation.cc:407] Connection header received
0807 06:33:27.589723 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:27.589725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:27.589739 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:27.589819 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:27.590976 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.591751 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.593224 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.593405 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.593461 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:27.595643 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:27.595645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:27.595645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:27.595653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:27.597329 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.597738 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.597740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.597740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.597804 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.598035 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.598036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.598036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.598169 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:27.598245 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:27.598468 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 06:33:27.598555 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:33:27.598698 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:27.588846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43095 (local address 127.2.243.2:35159)
0807 06:33:27.589048 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:27.589101 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:27.589601 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:27.589889 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:27.589894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:27.590681 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:27.590859 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:27.590863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.591847 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.591848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:27.593121 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:27.593124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.593489 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.593490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:27.593640 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:27.594133 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:27.594148 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:27.595553 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:27.597428 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:27.597434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:27.597435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:27.597661 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:27.597873 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:27.597873 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:27.597873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:27.597963 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:27.598238 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:27.598242 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:27.598343 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 06:33:27.598419 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:33:27.598555 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260807 06:33:27.599215 7421 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43095
I20260807 06:33:27.599316 7421 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
I20260807 06:33:27.599522 7421 heartbeater.cc:511] Master 127.2.243.62:43095 requested a full tablet report, sending...
I20260807 06:33:27.599910 6881 ts_manager.cc:194] Registered new tserver with Master: d4224727be0a495c80e321cd0f2834ca (127.2.243.2:33707)
I20260807 06:33:27.600235 6881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:35159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:27.617442 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43095
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:27.701162 7429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:27.701370 7429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:27.701393 7429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:27.702631 7429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:27.702678 7429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:27.702705 7429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:27.704177 7429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:27.704419 7429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:27.704666 7429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:27.711683 7444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:27.711678 7440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:27.711941 7429 server_base.cc:1034] running on GCE node
W20260807 06:33:27.711683 7442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:27.712193 7429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:27.712469 7429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:27.713661 7429 hybrid_clock.cc:648] HybridClock initialized: now 1786084407713653 us; error 46 us; skew 500 ppm
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:27.716121 7429 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:27.716693 7429 webserver.cc:460] Webserver started at http://127.2.243.3:43833/ using document root <none> and password file <none>
I20260807 06:33:27.716923 7429 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:27.716990 7429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:27.717139 7429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:27.718246 7429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "83999cda0c1c4a899cbbb37fd0e016f5"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "aad558102c1f2d46577f2cfe31eaf4cd"
server_key_iv: "595b2db5e599ad2430bdfbf7cfa36e38"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.718637 7429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "83999cda0c1c4a899cbbb37fd0e016f5"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "aad558102c1f2d46577f2cfe31eaf4cd"
server_key_iv: "595b2db5e599ad2430bdfbf7cfa36e38"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.720111 7429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:27.720847 7455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.721047 7429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:27.721141 7429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "83999cda0c1c4a899cbbb37fd0e016f5"
format_stamp: "Formatted at 2026-08-07 06:33:27 on dist-test-slave-rwrg"
server_key: "aad558102c1f2d46577f2cfe31eaf4cd"
server_key_iv: "595b2db5e599ad2430bdfbf7cfa36e38"
server_key_version: "encryptionkey@0"
I20260807 06:33:27.721195 7429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:27.746126 7429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:27.748624 7429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:27.748750 7429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:27.748956 7429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:27.749315 7429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:27.749353 7429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.749399 7429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:27.749425 7429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:27.755769 7429 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:46479
I20260807 06:33:27.755837 7662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:46479 every 8 connection(s)
I20260807 06:33:27.756151 7429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[6812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:27.762856 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 7429
I20260807 06:33:27.762961 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:27.763171 3020 external_mini_cluster.cc:1428] Setting key 80ff723a0635076c7d5506d41bc0dee7
I20260807 06:33:27.765921 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:27.756855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:59447 (local address 127.2.243.62:43095)
0807 06:33:27.756918 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:27.756920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:27.757128 (+ 208us) server_negotiation.cc:407] Connection header received
0807 06:33:27.757628 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:27.757634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:27.757647 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:27.757717 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:27.758878 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.759468 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.760466 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.760730 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.760785 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:27.762951 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:27.762954 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:27.762955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:27.762964 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:27.764478 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.764931 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.764932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.764933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.764989 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:27.765273 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:27.765274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:27.765275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:27.765396 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:27.765472 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:27.765750 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 06:33:27.765829 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:33:27.766135 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:27.756819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43095 (local address 127.2.243.3:59447)
0807 06:33:27.757006 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:27.757047 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:27.757516 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:27.757816 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:27.757819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:27.758493 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:27.758784 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:27.758790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.759563 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.759564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:27.760390 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:27.760392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:27.760802 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:27.760802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:27.760899 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:27.761398 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:27.761406 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:27.762829 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:27.764600 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:27.764607 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:27.764609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:27.764854 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:27.765067 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:27.765068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:27.765068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:27.765203 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:27.765462 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:27.765469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:27.765610 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 06:33:27.765716 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:33:27.765874 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260807 06:33:27.766582 7663 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43095
I20260807 06:33:27.766676 7663 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:27.766841 7663 heartbeater.cc:511] Master 127.2.243.62:43095 requested a full tablet report, sending...
I20260807 06:33:27.767194 6881 ts_manager.cc:194] Registered new tserver with Master: 83999cda0c1c4a899cbbb37fd0e016f5 (127.2.243.3:46479)
I20260807 06:33:27.767556 6881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:59447
I20260807 06:33:27.775050 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:33:27.787230 6964 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:27.786304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.2.243.62:43095)
0807 06:33:27.786385 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:27.786387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:27.786549 (+ 162us) server_negotiation.cc:407] Connection header received
0807 06:33:27.786576 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:27.786576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:27.786585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:27.786643 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:27.787155 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37666: BlockingRecv error: recv got EOF from 127.0.0.1:37666 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 06:33:27.788094 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6984
I20260807 06:33:27.800191 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 7201
I20260807 06:33:27.827780 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 7429
I20260807 06:33:27.837432 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6836
2026-08-07T06:33:27Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2126 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[7688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[7688](info): set up 2 sockets
Aug 07 06:33:27 dist-test-slave-rwrg krb5kdc[7688](info): commencing operation
krb5kdc: starting...
W20260807 06:33:29.195439 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260807 06:33:29.439949 3020 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.244s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260807 06:33:29.773511 3020 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.333s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260807 06:33:29.952745 3020 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.179s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:29 dist-test-slave-rwrg krb5kdc[7688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:30.017840 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33425
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:33425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:30.102231 7711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:30.102485 7711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:30.102515 7711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:30.103632 7711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:30.103668 7711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:30.103682 7711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:30.103695 7711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:30.103708 7711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:30.105206 7711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:33425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:30.105406 7711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:30.105592 7711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:30.110685 7722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:30.110772 7724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:30.110889 7727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:30.110889 7711 server_base.cc:1034] running on GCE node
I20260807 06:33:30.111209 7711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:30.111456 7711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:30.112609 7711 hybrid_clock.cc:648] HybridClock initialized: now 1786084410112579 us; error 42 us; skew 500 ppm
Aug 07 06:33:30 dist-test-slave-rwrg krb5kdc[7688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084410, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:30.115497 7711 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:30.116017 7711 webserver.cc:460] Webserver started at http://127.2.243.62:36729/ using document root <none> and password file <none>
I20260807 06:33:30.116288 7711 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:30.116361 7711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:30.116566 7711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:30.117760 7711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "cc4b2a2dce7a48f89d47264eb061525a"
format_stamp: "Formatted at 2026-08-07 06:33:30 on dist-test-slave-rwrg"
server_key: "ad97c53975b921ab80c373d68499765d"
server_key_iv: "9d8145235eb38a5f9c6bd53fca80b6a5"
server_key_version: "encryptionkey@0"
I20260807 06:33:30.118151 7711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "cc4b2a2dce7a48f89d47264eb061525a"
format_stamp: "Formatted at 2026-08-07 06:33:30 on dist-test-slave-rwrg"
server_key: "ad97c53975b921ab80c373d68499765d"
server_key_iv: "9d8145235eb38a5f9c6bd53fca80b6a5"
server_key_version: "encryptionkey@0"
I20260807 06:33:30.119620 7711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:30.120415 7744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:30.120723 7711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:30.120834 7711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "cc4b2a2dce7a48f89d47264eb061525a"
format_stamp: "Formatted at 2026-08-07 06:33:30 on dist-test-slave-rwrg"
server_key: "ad97c53975b921ab80c373d68499765d"
server_key_iv: "9d8145235eb38a5f9c6bd53fca80b6a5"
server_key_version: "encryptionkey@0"
I20260807 06:33:30.120908 7711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:30.145576 7711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:30.147861 7711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:30.148034 7711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:30.152354 7711 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:33425
I20260807 06:33:30.152390 7822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:33425 every 8 connection(s)
I20260807 06:33:30.152806 7711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:30.153635 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 7711
I20260807 06:33:30.153731 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:30.153882 7823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:30.153939 3020 external_mini_cluster.cc:1428] Setting key 87bdef135f930b81aae959fcaeb35c77
I20260807 06:33:30.157436 7823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a: Bootstrap starting.
I20260807 06:33:30.158185 7823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:30.158468 7823 log.cc:824] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:30.159189 7823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a: No bootstrap required, opened a new log
Aug 07 06:33:30 dist-test-slave-rwrg krb5kdc[7688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:30.161309 7823 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4b2a2dce7a48f89d47264eb061525a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33425 } }
I20260807 06:33:30.161494 7823 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:30.161552 7823 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4b2a2dce7a48f89d47264eb061525a, State: Initialized, Role: FOLLOWER
I20260807 06:33:30.161696 7823 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4b2a2dce7a48f89d47264eb061525a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33425 } }
I20260807 06:33:30.161783 7823 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:30.161837 7823 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:30.161887 7823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:30.162575 7823 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4b2a2dce7a48f89d47264eb061525a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33425 } }
I20260807 06:33:30.162706 7823 leader_election.cc:302] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc4b2a2dce7a48f89d47264eb061525a; no voters:
I20260807 06:33:30.162881 7823 leader_election.cc:288] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:30.162943 7829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:30.163106 7829 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [term 1 LEADER]: Becoming Leader. State: Replica: cc4b2a2dce7a48f89d47264eb061525a, State: Running, Role: LEADER
I20260807 06:33:30.163172 7823 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:30.163203 7829 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4b2a2dce7a48f89d47264eb061525a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33425 } }
I20260807 06:33:30.163640 7830 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc4b2a2dce7a48f89d47264eb061525a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4b2a2dce7a48f89d47264eb061525a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33425 } } }
I20260807 06:33:30.163738 7830 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:30.163626 7829 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc4b2a2dce7a48f89d47264eb061525a. Latest consensus state: current_term: 1 leader_uuid: "cc4b2a2dce7a48f89d47264eb061525a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4b2a2dce7a48f89d47264eb061525a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33425 } } }
I20260807 06:33:30.163973 7829 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:30.164099 7844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:30.164683 7844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:30.165804 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:30.154468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60620 (local address 127.2.243.62:33425)
0807 06:33:30.154646 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:30.154655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:30.154687 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:33:30.155067 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:30.155074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:30.155119 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:30.155232 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:30.156100 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:30.157090 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:30.158134 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:30.158746 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:30.159011 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:30.161746 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:30.161754 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:30.161755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:30.161766 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:30.163810 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:30.164292 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:30.164295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:30.164296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:30.164412 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:30.164662 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:30.164663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:30.164663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:30.164969 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:30.165134 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:30.165474 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 06:33:30.165606 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260807 06:33:30.166324 7844 catalog_manager.cc:1347] Generated new cluster ID: dec77d51c651404c815a90b05818b046
I20260807 06:33:30.166371 7844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:30.166769 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:33:30.173868 7844 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:30.174368 7844 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:30.178956 7844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc4b2a2dce7a48f89d47264eb061525a: Generated new TSK 0
I20260807 06:33:30.179140 7844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:33:30.191196 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:30.187698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.2.243.62:33425)
0807 06:33:30.187763 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:30.187766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:30.187778 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:30.187829 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:30.187831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:30.187842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:30.187915 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:30.188306 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:30.188904 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:30.189680 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:30.189893 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:30.189968 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:30.190703 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:30.190704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:30.190705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:30.190891 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:30.190907 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:30.190913 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:30.190976 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:30.191071 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 06:33:30.191098 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:33:30.193976 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 7711
2026-08-07T06:33:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2359 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:30 dist-test-slave-rwrg krb5kdc[7875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:30 dist-test-slave-rwrg krb5kdc[7875](info): set up 2 sockets
Aug 07 06:33:30 dist-test-slave-rwrg krb5kdc[7875](info): commencing operation
krb5kdc: starting...
W20260807 06:33:31.469974 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:31 dist-test-slave-rwrg krb5kdc[7875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:31.595528 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:36617
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:36617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:31.680127 7893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:31.680342 7893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:31.680365 7893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:31.681576 7893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:31.681622 7893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:31.681636 7893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:31.681649 7893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:31.681706 7893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:31.683179 7893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:36617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:36617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:31.683440 7893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:31.683681 7893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:31.688671 7904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:31.688758 7899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:31.688668 7901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:31.689035 7893 server_base.cc:1034] running on GCE node
I20260807 06:33:31.689294 7893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:31.689535 7893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:31.690670 7893 hybrid_clock.cc:648] HybridClock initialized: now 1786084411690639 us; error 47 us; skew 500 ppm
Aug 07 06:33:31 dist-test-slave-rwrg krb5kdc[7875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084411, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:31.693218 7893 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:31.693603 7893 webserver.cc:460] Webserver started at http://127.2.243.62:43225/ using document root <none> and password file <none>
I20260807 06:33:31.693801 7893 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:31.693843 7893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:31.693933 7893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:31.694902 7893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "00cffc8648b348df9301ab5781efc72a"
format_stamp: "Formatted at 2026-08-07 06:33:31 on dist-test-slave-rwrg"
server_key: "bda0d52399dd4ad2ca69514f30604069"
server_key_iv: "04ac905d0750a8d0bc91af1888a08620"
server_key_version: "encryptionkey@0"
I20260807 06:33:31.695209 7893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "00cffc8648b348df9301ab5781efc72a"
format_stamp: "Formatted at 2026-08-07 06:33:31 on dist-test-slave-rwrg"
server_key: "bda0d52399dd4ad2ca69514f30604069"
server_key_iv: "04ac905d0750a8d0bc91af1888a08620"
server_key_version: "encryptionkey@0"
I20260807 06:33:31.696452 7893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:33:31.697243 7913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:31.697428 7893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:31.697523 7893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "00cffc8648b348df9301ab5781efc72a"
format_stamp: "Formatted at 2026-08-07 06:33:31 on dist-test-slave-rwrg"
server_key: "bda0d52399dd4ad2ca69514f30604069"
server_key_iv: "04ac905d0750a8d0bc91af1888a08620"
server_key_version: "encryptionkey@0"
I20260807 06:33:31.697594 7893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:31.719193 7893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:31.721444 7893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:31.721612 7893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:31.725695 7893 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:36617
I20260807 06:33:31.725740 8011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:36617 every 8 connection(s)
I20260807 06:33:31.726130 7893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:31.727301 8012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:31.729911 8012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a: Bootstrap starting.
I20260807 06:33:31.730561 8012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:31.730865 8012 log.cc:824] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:31.731278 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 7893
I20260807 06:33:31.731384 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:31.731591 3020 external_mini_cluster.cc:1428] Setting key 978aff09b3f760f8e0437b651a4a6a43
I20260807 06:33:31.731786 8012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a: No bootstrap required, opened a new log
I20260807 06:33:31.733652 8012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cffc8648b348df9301ab5781efc72a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36617 } }
I20260807 06:33:31.733789 8012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:31.733812 8012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cffc8648b348df9301ab5781efc72a, State: Initialized, Role: FOLLOWER
I20260807 06:33:31.733966 8012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cffc8648b348df9301ab5781efc72a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36617 } }
I20260807 06:33:31.734047 8012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:31.734073 8012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:31.734125 8012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:31.734943 8012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cffc8648b348df9301ab5781efc72a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36617 } }
I20260807 06:33:31.735064 8012 leader_election.cc:302] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00cffc8648b348df9301ab5781efc72a; no voters:
I20260807 06:33:31.735221 8012 leader_election.cc:288] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:31.735278 8017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:31.735481 8012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:31.735502 8017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [term 1 LEADER]: Becoming Leader. State: Replica: 00cffc8648b348df9301ab5781efc72a, State: Running, Role: LEADER
I20260807 06:33:31.735718 8017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cffc8648b348df9301ab5781efc72a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36617 } }
I20260807 06:33:31.736111 8018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00cffc8648b348df9301ab5781efc72a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cffc8648b348df9301ab5781efc72a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36617 } } }
I20260807 06:33:31.736130 8020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00cffc8648b348df9301ab5781efc72a. Latest consensus state: current_term: 1 leader_uuid: "00cffc8648b348df9301ab5781efc72a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cffc8648b348df9301ab5781efc72a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36617 } } }
I20260807 06:33:31.736219 8018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:31.736253 8020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:31.736442 8030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:31.737035 8030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:31.738837 8030 catalog_manager.cc:1347] Generated new cluster ID: a01c9078322645ee987a98b19742e0b9
I20260807 06:33:31.738884 8030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:31 dist-test-slave-rwrg krb5kdc[7875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:31.744473 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:31.732181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46940 (local address 127.2.243.62:36617)
0807 06:33:31.732416 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:31.732427 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:31.732535 (+ 108us) server_negotiation.cc:407] Connection header received
0807 06:33:31.733645 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:31.733653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:31.733704 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:31.733873 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:31.734681 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:31.736047 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:31.737630 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:31.738252 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:31.738380 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:31.740951 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:31.740955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:31.740955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:31.740968 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:31.742799 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:31.743226 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:31.743229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:31.743229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:31.743341 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:31.743574 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:31.743576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:31.743576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:31.743821 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:31.744010 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:31.744113 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 06:33:31.744260 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":96,"threads_started":1}
I20260807 06:33:31.749437 8030 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:31.750138 8030 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:31.752643 8030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00cffc8648b348df9301ab5781efc72a: Generated new TSK 0
I20260807 06:33:31.752826 8030 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:31.783690 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36617
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:31.890041 8049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:31.890251 8049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:31.890276 8049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:31.891428 8049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:31.891470 8049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:31.891496 8049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:31.892989 8049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:31.893250 8049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:31.893477 8049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:31.899189 8056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:31.899210 8055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:31.899483 8049 server_base.cc:1034] running on GCE node
W20260807 06:33:31.899231 8060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:31.899801 8049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:31.900053 8049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:31.901225 8049 hybrid_clock.cc:648] HybridClock initialized: now 1786084411901186 us; error 54 us; skew 500 ppm
Aug 07 06:33:31 dist-test-slave-rwrg krb5kdc[7875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084411, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:31.903978 8049 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:31.904412 8049 webserver.cc:460] Webserver started at http://127.2.243.1:44739/ using document root <none> and password file <none>
I20260807 06:33:31.904681 8049 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:31.904729 8049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:31.904868 8049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:31.905956 8049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "9dc78075115745a2bd3c979286adb6fc"
format_stamp: "Formatted at 2026-08-07 06:33:31 on dist-test-slave-rwrg"
server_key: "b8c530ee76247c5a638bca6438222c9d"
server_key_iv: "132633843dd80bd98c3704bf25bc82f7"
server_key_version: "encryptionkey@0"
I20260807 06:33:31.906328 8049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "9dc78075115745a2bd3c979286adb6fc"
format_stamp: "Formatted at 2026-08-07 06:33:31 on dist-test-slave-rwrg"
server_key: "b8c530ee76247c5a638bca6438222c9d"
server_key_iv: "132633843dd80bd98c3704bf25bc82f7"
server_key_version: "encryptionkey@0"
I20260807 06:33:31.907687 8049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:31.908438 8071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:31.908679 8049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:31.908788 8049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "9dc78075115745a2bd3c979286adb6fc"
format_stamp: "Formatted at 2026-08-07 06:33:31 on dist-test-slave-rwrg"
server_key: "b8c530ee76247c5a638bca6438222c9d"
server_key_iv: "132633843dd80bd98c3704bf25bc82f7"
server_key_version: "encryptionkey@0"
I20260807 06:33:31.908861 8049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:31.922093 8049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:31.924005 8049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:31.924183 8049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:31.924409 8049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:31.924754 8049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:31.924809 8049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:31.924851 8049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:31.924880 8049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:31.931241 8049 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:44923
I20260807 06:33:31.931295 8258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:44923 every 8 connection(s)
I20260807 06:33:31.931604 8049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:33:31 dist-test-slave-rwrg krb5kdc[7875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084411, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:31.939898 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 8049
I20260807 06:33:31.939985 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:31.940191 3020 external_mini_cluster.cc:1428] Setting key 92ef1ac45c0e567049a1e04e120806b7
I20260807 06:33:31.940802 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:31.932310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:43307 (local address 127.2.243.62:36617)
0807 06:33:31.932380 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:31.932383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:31.932632 (+ 249us) server_negotiation.cc:407] Connection header received
0807 06:33:31.933105 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:31.933106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:31.933117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:31.933160 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:31.934082 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:31.934592 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:31.935608 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:31.935827 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:31.935880 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:31.938097 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:31.938098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:31.938099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:31.938107 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:31.939297 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:31.939730 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:31.939731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:31.939731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:31.939793 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:31.940101 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:31.940102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:31.940102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:31.940217 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:31.940295 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:31.940616 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 06:33:31.940708 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:33:31.940956 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:31.932292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:36617 (local address 127.2.243.1:43307)
0807 06:33:31.932468 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:31.932546 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:31.933012 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:31.933239 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:31.933242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:31.933815 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:31.933996 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:31.934001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:31.934671 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:31.934672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:31.935520 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:31.935522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:31.935898 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:31.935899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:31.936004 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:31.936602 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:31.936609 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:31.937987 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:31.939403 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:31.939410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:31.939412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:31.939638 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:31.939862 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:31.939863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:31.939863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:31.940023 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:31.940300 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:31.940306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:31.940455 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 06:33:31.940583 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 06:33:31.940718 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260807 06:33:31.942801 8259 heartbeater.cc:347] Connected to a master server at 127.2.243.62:36617
I20260807 06:33:31.942916 8259 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:31.943099 8259 heartbeater.cc:511] Master 127.2.243.62:36617 requested a full tablet report, sending...
I20260807 06:33:31.943650 7952 ts_manager.cc:194] Registered new tserver with Master: 9dc78075115745a2bd3c979286adb6fc (127.2.243.1:44923)
I20260807 06:33:31.944600 7952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:43307
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:31.967731 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36617
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:32.049638 8264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:32.049860 8264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:32.049885 8264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:32.051014 8264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:32.051054 8264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:32.051081 8264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:32.052479 8264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:32.052695 8264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:32.052881 8264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:32.057886 8276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:32.057950 8281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:32.057997 8279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:32.058033 8264 server_base.cc:1034] running on GCE node
I20260807 06:33:32.058418 8264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:32.058681 8264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:32.059846 8264 hybrid_clock.cc:648] HybridClock initialized: now 1786084412059829 us; error 40 us; skew 500 ppm
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[7875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:32.062455 8264 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:32.062948 8264 webserver.cc:460] Webserver started at http://127.2.243.2:44837/ using document root <none> and password file <none>
I20260807 06:33:32.063184 8264 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:32.063251 8264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:32.063388 8264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:32.064424 8264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "d7b2a6a4f5ee4d56b323d54d64bd1ed9"
format_stamp: "Formatted at 2026-08-07 06:33:32 on dist-test-slave-rwrg"
server_key: "3e1da1947a5d8a88f54cbbac50dfd4f4"
server_key_iv: "7aecd488c3541d75284be6b340728aab"
server_key_version: "encryptionkey@0"
I20260807 06:33:32.064857 8264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "d7b2a6a4f5ee4d56b323d54d64bd1ed9"
format_stamp: "Formatted at 2026-08-07 06:33:32 on dist-test-slave-rwrg"
server_key: "3e1da1947a5d8a88f54cbbac50dfd4f4"
server_key_iv: "7aecd488c3541d75284be6b340728aab"
server_key_version: "encryptionkey@0"
I20260807 06:33:32.066211 8264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:32.066928 8292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:32.067094 8264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:32.067193 8264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "d7b2a6a4f5ee4d56b323d54d64bd1ed9"
format_stamp: "Formatted at 2026-08-07 06:33:32 on dist-test-slave-rwrg"
server_key: "3e1da1947a5d8a88f54cbbac50dfd4f4"
server_key_iv: "7aecd488c3541d75284be6b340728aab"
server_key_version: "encryptionkey@0"
I20260807 06:33:32.067289 8264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:32.079761 8264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:32.081609 8264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:32.081770 8264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:32.081993 8264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:32.082298 8264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:32.082352 8264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:32.082393 8264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:32.082422 8264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:32.088380 8264 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:44101
I20260807 06:33:32.088421 8478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:44101 every 8 connection(s)
I20260807 06:33:32.088816 8264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:32.092969 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 8264
I20260807 06:33:32.093083 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:32.093291 3020 external_mini_cluster.cc:1428] Setting key 14378bbe5077a0a2df6691867af5fede
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[7875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:32.099081 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:32.089574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:60571 (local address 127.2.243.62:36617)
0807 06:33:32.089625 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:32.089627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:32.089825 (+ 198us) server_negotiation.cc:407] Connection header received
0807 06:33:32.090312 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:32.090314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:32.090324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:32.090366 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:32.091317 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.091821 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.092890 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.093202 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.093283 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:32.095920 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:32.095921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:32.095922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:32.095930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:32.097526 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:32.097997 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:32.097998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:32.097999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:32.098072 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:32.098351 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:32.098352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:32.098353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:32.098484 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:32.098572 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:32.098860 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 06:33:32.098986 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:33:32.099239 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:32.089542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:36617 (local address 127.2.243.2:60571)
0807 06:33:32.089691 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:32.089732 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:32.090201 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:32.090418 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:32.090421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:32.091014 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:32.091197 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:32.091201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.091912 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.091913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:32.092801 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:32.092804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.093287 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.093288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:32.093424 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:32.093952 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:32.093962 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:32.095798 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:32.097643 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:32.097651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:32.097654 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:32.097905 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:32.098158 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:32.098159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:32.098160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:32.098276 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:32.098562 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:32.098569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:32.098718 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 06:33:32.098838 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 06:33:32.099010 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260807 06:33:32.099848 8479 heartbeater.cc:347] Connected to a master server at 127.2.243.62:36617
I20260807 06:33:32.099970 8479 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:32.100152 8479 heartbeater.cc:511] Master 127.2.243.62:36617 requested a full tablet report, sending...
I20260807 06:33:32.100616 7952 ts_manager.cc:194] Registered new tserver with Master: d7b2a6a4f5ee4d56b323d54d64bd1ed9 (127.2.243.2:44101)
I20260807 06:33:32.101182 7952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:60571
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:32.122843 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36617
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:32.205845 8486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:32.206058 8486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:32.206080 8486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:32.207159 8486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:32.207195 8486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:32.207221 8486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:32.208740 8486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:32.208932 8486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:32.209110 8486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:32.214102 8499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:32.214120 8498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:32.214102 8504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:32.214344 8486 server_base.cc:1034] running on GCE node
I20260807 06:33:32.214574 8486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:32.214835 8486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:32.215998 8486 hybrid_clock.cc:648] HybridClock initialized: now 1786084412215985 us; error 44 us; skew 500 ppm
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[7875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:32.221791 8486 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:32.222208 8486 webserver.cc:460] Webserver started at http://127.2.243.3:33193/ using document root <none> and password file <none>
I20260807 06:33:32.222474 8486 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:32.222551 8486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:32.222710 8486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:32.223793 8486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "b1b8a2a36abb4f79b8b1af685a3a78d0"
format_stamp: "Formatted at 2026-08-07 06:33:32 on dist-test-slave-rwrg"
server_key: "f480684c6f616591ce39de90dc16ce05"
server_key_iv: "70d5e7df8919d98f38e8ce7989386257"
server_key_version: "encryptionkey@0"
I20260807 06:33:32.224148 8486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "b1b8a2a36abb4f79b8b1af685a3a78d0"
format_stamp: "Formatted at 2026-08-07 06:33:32 on dist-test-slave-rwrg"
server_key: "f480684c6f616591ce39de90dc16ce05"
server_key_iv: "70d5e7df8919d98f38e8ce7989386257"
server_key_version: "encryptionkey@0"
I20260807 06:33:32.225605 8486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 06:33:32.226378 8516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:32.226593 8486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:32.226701 8486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "b1b8a2a36abb4f79b8b1af685a3a78d0"
format_stamp: "Formatted at 2026-08-07 06:33:32 on dist-test-slave-rwrg"
server_key: "f480684c6f616591ce39de90dc16ce05"
server_key_iv: "70d5e7df8919d98f38e8ce7989386257"
server_key_version: "encryptionkey@0"
I20260807 06:33:32.226771 8486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:32.262058 8486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:32.264457 8486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:32.264652 8486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:32.264889 8486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:32.265203 8486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:32.265259 8486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:32.265298 8486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:32.265326 8486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:32.271356 8486 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:46487
I20260807 06:33:32.271400 8714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:46487 every 8 connection(s)
I20260807 06:33:32.271756 8486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[7875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:32.278692 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 8486
I20260807 06:33:32.278797 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:32.278983 3020 external_mini_cluster.cc:1428] Setting key deaa4266454b4fbbe413f4baf63ce42f
I20260807 06:33:32.281322 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:32.272510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:38107 (local address 127.2.243.62:36617)
0807 06:33:32.272575 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:32.272578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:32.272792 (+ 214us) server_negotiation.cc:407] Connection header received
0807 06:33:32.273313 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:32.273314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:32.273325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:32.273385 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:32.274297 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.274799 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.275779 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.275983 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.276084 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:32.278225 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:32.278227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:32.278227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:32.278234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:32.279982 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:32.280361 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:32.280362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:32.280362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:32.280424 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:32.280751 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:32.280752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:32.280753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:32.280865 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:32.280923 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:32.281185 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 06:33:32.281260 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:33:32.281376 8721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:32.272476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:36617 (local address 127.2.243.3:38107)
0807 06:33:32.272672 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:32.272706 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:32.273205 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:32.273449 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:32.273455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:32.274012 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:32.274196 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:32.274200 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.274893 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.274895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:32.275682 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:32.275684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.276094 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.276095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:32.276221 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:32.276820 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:32.276827 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:32.278127 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:32.280057 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:32.280063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:32.280065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:32.280271 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:32.280593 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:32.280593 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:32.280594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:32.280689 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:32.280925 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:32.280929 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:32.281035 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 06:33:32.281113 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:33:32.281225 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":120,"threads_started":1}
I20260807 06:33:32.281847 8718 heartbeater.cc:347] Connected to a master server at 127.2.243.62:36617
I20260807 06:33:32.281949 8718 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:32.282135 8718 heartbeater.cc:511] Master 127.2.243.62:36617 requested a full tablet report, sending...
I20260807 06:33:32.282459 7952 ts_manager.cc:194] Registered new tserver with Master: b1b8a2a36abb4f79b8b1af685a3a78d0 (127.2.243.3:46487)
I20260807 06:33:32.282819 7952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:38107
I20260807 06:33:32.290694 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[7875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084412, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[7875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084412, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:32.316013 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:32.307649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46944 (local address 127.2.243.62:36617)
0807 06:33:32.307761 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:32.307764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:32.307886 (+ 122us) server_negotiation.cc:407] Connection header received
0807 06:33:32.307984 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:32.307985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:32.308002 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:32.308093 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:32.308742 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.309247 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.310003 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:32.310190 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:32.310265 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:32.312874 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:32.312875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:32.312876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:32.312883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:32.314729 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:32.315148 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:32.315149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:32.315150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:32.315210 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:32.315418 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:32.315419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:32.315419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:32.315673 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:32.315762 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:32.315867 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 06:33:32.315927 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 06:33:32.316272 7952 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46944
I20260807 06:33:32.316715 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 8049
I20260807 06:33:32.334168 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 8264
I20260807 06:33:32.348240 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 8486
I20260807 06:33:32.356709 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 7893
2026-08-07T06:33:32Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2159 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[8733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[8733](info): set up 2 sockets
Aug 07 06:33:32 dist-test-slave-rwrg krb5kdc[8733](info): commencing operation
krb5kdc: starting...
W20260807 06:33:33.618291 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:33 dist-test-slave-rwrg krb5kdc[8733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:33.743269 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39793
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:33.824376 8760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:33.824651 8760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:33.824679 8760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:33.825745 8760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:33.825781 8760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:33.825795 8760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:33.825807 8760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:33.825819 8760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:33.827234 8760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:33.827461 8760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:33.827657 8760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:33.832438 8776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:33.832454 8780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:33.832628 8760 server_base.cc:1034] running on GCE node
W20260807 06:33:33.832443 8777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:33.832928 8760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:33.833230 8760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:33.834406 8760 hybrid_clock.cc:648] HybridClock initialized: now 1786084413834387 us; error 55 us; skew 500 ppm
Aug 07 06:33:33 dist-test-slave-rwrg krb5kdc[8733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084413, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:33.836980 8760 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:33.837440 8760 webserver.cc:460] Webserver started at http://127.2.243.62:37087/ using document root <none> and password file <none>
I20260807 06:33:33.837710 8760 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:33.837756 8760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:33.837845 8760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:33.838840 8760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2"
format_stamp: "Formatted at 2026-08-07 06:33:33 on dist-test-slave-rwrg"
server_key: "2921e6c9a5598651e928b082d5fc1625"
server_key_iv: "3d41de3692bb1ecbf5f61a2223abd2eb"
server_key_version: "encryptionkey@0"
I20260807 06:33:33.839138 8760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2"
format_stamp: "Formatted at 2026-08-07 06:33:33 on dist-test-slave-rwrg"
server_key: "2921e6c9a5598651e928b082d5fc1625"
server_key_iv: "3d41de3692bb1ecbf5f61a2223abd2eb"
server_key_version: "encryptionkey@0"
I20260807 06:33:33.840575 8760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:33.841346 8791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:33.841566 8760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:33.841643 8760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2"
format_stamp: "Formatted at 2026-08-07 06:33:33 on dist-test-slave-rwrg"
server_key: "2921e6c9a5598651e928b082d5fc1625"
server_key_iv: "3d41de3692bb1ecbf5f61a2223abd2eb"
server_key_version: "encryptionkey@0"
I20260807 06:33:33.841696 8760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:33.853458 8760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:33.855418 8760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:33.855527 8760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:33.859758 8760 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39793
I20260807 06:33:33.859835 8883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39793 every 8 connection(s)
I20260807 06:33:33.860435 8760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:33.861222 8884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:33.863523 8884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2: Bootstrap starting.
I20260807 06:33:33.864128 8884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:33.864432 8884 log.cc:824] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:33.865159 8884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2: No bootstrap required, opened a new log
I20260807 06:33:33.866703 8884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39793 } }
I20260807 06:33:33.866837 8884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:33.866863 8884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 674b29d0ece447fe9a12fa3fd1a9c9e2, State: Initialized, Role: FOLLOWER
I20260807 06:33:33.867048 8884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39793 } }
I20260807 06:33:33.867112 8884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:33.867136 8884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:33.867203 8884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:33.868083 8884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39793 } }
I20260807 06:33:33.868119 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 8760
I20260807 06:33:33.868232 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:33.868209 8884 leader_election.cc:302] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 674b29d0ece447fe9a12fa3fd1a9c9e2; no voters:
I20260807 06:33:33.868407 3020 external_mini_cluster.cc:1428] Setting key 030bcce38f73ac7bc3029aa8ffd63c0f
I20260807 06:33:33.868407 8884 leader_election.cc:288] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:33.868439 8894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:33.868599 8894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [term 1 LEADER]: Becoming Leader. State: Replica: 674b29d0ece447fe9a12fa3fd1a9c9e2, State: Running, Role: LEADER
I20260807 06:33:33.868706 8884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:33.868752 8894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39793 } }
I20260807 06:33:33.869065 8894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 674b29d0ece447fe9a12fa3fd1a9c9e2. Latest consensus state: current_term: 1 leader_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39793 } } }
I20260807 06:33:33.869081 8900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "674b29d0ece447fe9a12fa3fd1a9c9e2" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39793 } } }
I20260807 06:33:33.869263 8894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:33.869517 8900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:33.870076 8914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:33.870679 8914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:33.872331 8914 catalog_manager.cc:1347] Generated new cluster ID: a8e0c2f3077c4c8abaea733f4933c7d2
I20260807 06:33:33.872385 8914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:33 dist-test-slave-rwrg krb5kdc[8733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:33.884673 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:33.868990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44282 (local address 127.2.243.62:39793)
0807 06:33:33.869204 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:33.869228 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:33.869488 (+ 260us) server_negotiation.cc:407] Connection header received
0807 06:33:33.869614 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:33.869622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:33.869682 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:33.869837 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:33.870638 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:33.872012 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:33.877385 (+ 5373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:33.878010 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:33.878109 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:33.881028 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:33.881033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:33.881034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:33.881047 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:33.883014 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:33.883399 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:33.883402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:33.883403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:33.883519 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:33.883714 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:33.883715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:33.883715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:33.884013 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:33.884176 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:33.884244 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 06:33:33.884396 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
I20260807 06:33:33.885596 8914 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:33.886065 8914 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:33.890187 8914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 674b29d0ece447fe9a12fa3fd1a9c9e2: Generated new TSK 0
I20260807 06:33:33.890367 8914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:33.921679 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39793
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:34.004154 8933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:34.004395 8933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:34.004436 8933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:34.005605 8933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:34.005676 8933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:34.005723 8933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:34.007308 8933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:34.007568 8933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:34.007797 8933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:34.012773 8940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:34.012871 8933 server_base.cc:1034] running on GCE node
W20260807 06:33:34.012948 8943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:34.012810 8939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:34.013242 8933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:34.013505 8933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:34.014691 8933 hybrid_clock.cc:648] HybridClock initialized: now 1786084414014683 us; error 51 us; skew 500 ppm
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:34.017176 8933 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:34.017554 8933 webserver.cc:460] Webserver started at http://127.2.243.1:35981/ using document root <none> and password file <none>
I20260807 06:33:34.017729 8933 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:34.017768 8933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:34.017853 8933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:34.018811 8933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "959ccc23f76b46b4b6f92afbd43f0723"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "6d4fef6127330576f6a830b564c3ccf5"
server_key_iv: "04858ae511e4f184e01d78bf68a3021f"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.019107 8933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "959ccc23f76b46b4b6f92afbd43f0723"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "6d4fef6127330576f6a830b564c3ccf5"
server_key_iv: "04858ae511e4f184e01d78bf68a3021f"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.020335 8933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:34.021117 8955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.021291 8933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.021394 8933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "959ccc23f76b46b4b6f92afbd43f0723"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "6d4fef6127330576f6a830b564c3ccf5"
server_key_iv: "04858ae511e4f184e01d78bf68a3021f"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.021476 8933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:34.039994 8933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:34.042156 8933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:34.042333 8933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:34.042593 8933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:34.042912 8933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:34.042968 8933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.043005 8933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:34.043058 8933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.049362 8933 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:35631
I20260807 06:33:34.049450 9133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:35631 every 8 connection(s)
I20260807 06:33:34.049846 8933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:34.057566 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 8933
I20260807 06:33:34.057659 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:34.057858 3020 external_mini_cluster.cc:1428] Setting key 4765c54b0d192f5cdc821a9f4ee9e6df
I20260807 06:33:34.059405 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.050687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:60579 (local address 127.2.243.62:39793)
0807 06:33:34.050770 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:34.050772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:34.050908 (+ 136us) server_negotiation.cc:407] Connection header received
0807 06:33:34.051869 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:34.051870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:34.051883 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:34.051954 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:34.052608 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.053104 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.054096 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.054326 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.054394 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.056448 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:34.056449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:34.056450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:34.056456 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:34.057833 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.058457 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.058458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.058458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.058510 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.058765 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.058766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.058767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.058879 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:34.058949 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:34.059222 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 06:33:34.059318 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:33:34.059633 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.050640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39793 (local address 127.2.243.1:60579)
0807 06:33:34.050794 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:34.050834 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:34.051755 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:34.052010 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:34.052013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:34.052261 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:34.052445 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:34.052449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.053183 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.053184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:34.053999 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:34.054001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.054415 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.054416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:34.054519 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.055005 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:34.055013 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:34.056362 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:34.057922 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:34.057930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:34.057932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:34.058188 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:34.058587 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:34.058588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:34.058589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:34.058695 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:34.058951 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:34.058958 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:34.059087 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 06:33:34.059187 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:33:34.059340 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260807 06:33:34.060051 9134 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39793
I20260807 06:33:34.060166 9134 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:34.060339 9134 heartbeater.cc:511] Master 127.2.243.62:39793 requested a full tablet report, sending...
I20260807 06:33:34.060858 8825 ts_manager.cc:194] Registered new tserver with Master: 959ccc23f76b46b4b6f92afbd43f0723 (127.2.243.1:35631)
I20260807 06:33:34.061465 8825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:60579
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:34.083720 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39793
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:34.165714 9147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:34.165904 9147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:34.165926 9147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:34.167006 9147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:34.167044 9147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:34.167069 9147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:34.168586 9147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:34.168818 9147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:34.169049 9147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:34.174103 9157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:34.174119 9158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:34.174116 9162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:34.174350 9147 server_base.cc:1034] running on GCE node
I20260807 06:33:34.174486 9147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:34.174660 9147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:34.175800 9147 hybrid_clock.cc:648] HybridClock initialized: now 1786084414175788 us; error 29 us; skew 500 ppm
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:34.178309 9147 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:34.178791 9147 webserver.cc:460] Webserver started at http://127.2.243.2:33425/ using document root <none> and password file <none>
I20260807 06:33:34.179018 9147 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:34.179082 9147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:34.179210 9147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:34.180214 9147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "9263bef1c7ec4e32b09d5385c4587bec"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "ae6007f5594e8571af30cb63fd0bab3a"
server_key_iv: "0c1ccf987868a1dbd0c799d04f5c4886"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.180631 9147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "9263bef1c7ec4e32b09d5385c4587bec"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "ae6007f5594e8571af30cb63fd0bab3a"
server_key_iv: "0c1ccf987868a1dbd0c799d04f5c4886"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.182042 9147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:34.182862 9169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.183079 9147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:34.183176 9147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "9263bef1c7ec4e32b09d5385c4587bec"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "ae6007f5594e8571af30cb63fd0bab3a"
server_key_iv: "0c1ccf987868a1dbd0c799d04f5c4886"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.183261 9147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:34.196413 9147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:34.198251 9147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:34.198412 9147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:34.198637 9147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:34.198937 9147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:34.199014 9147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.199055 9147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:34.199082 9147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.205220 9147 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:41219
I20260807 06:33:34.205312 9331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:41219 every 8 connection(s)
I20260807 06:33:34.205638 9147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:34.209008 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 9147
I20260807 06:33:34.209152 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:34.209389 3020 external_mini_cluster.cc:1428] Setting key 844a2ddf7364af5b851ae149d7218110
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:34.215840 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.206372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:37295 (local address 127.2.243.62:39793)
0807 06:33:34.206416 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:34.206418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:34.206660 (+ 242us) server_negotiation.cc:407] Connection header received
0807 06:33:34.207611 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:34.207613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:34.207626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:34.207673 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:34.208276 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.208852 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.210058 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.210258 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.210348 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.212637 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:34.212639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:34.212639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:34.212647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:34.214174 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.214777 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.214779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.214779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.214859 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.215172 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.215173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.215173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.215283 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:34.215339 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:34.215695 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 06:33:34.215776 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:34.216168 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.206350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39793 (local address 127.2.243.2:37295)
0807 06:33:34.206537 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:34.206572 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:34.207490 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:34.207749 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:34.207753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:34.208005 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:34.208182 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:34.208186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.208946 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.208947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:34.209944 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:34.209947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.210371 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.210373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:34.210498 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.211021 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:34.211029 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:34.212457 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:34.214258 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:34.214267 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:34.214269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:34.214651 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:34.214947 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:34.214948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:34.214948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:34.215098 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:34.215356 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:34.215363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:34.215546 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 06:33:34.215677 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 06:33:34.215862 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260807 06:33:34.216753 9333 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39793
I20260807 06:33:34.216843 9333 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:34.217043 9333 heartbeater.cc:511] Master 127.2.243.62:39793 requested a full tablet report, sending...
I20260807 06:33:34.217481 8825 ts_manager.cc:194] Registered new tserver with Master: 9263bef1c7ec4e32b09d5385c4587bec (127.2.243.2:41219)
I20260807 06:33:34.217970 8825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:37295
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:34.239261 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39793
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:34.319713 9339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:34.319995 9339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:34.320037 9339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:34.321525 9339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:34.321599 9339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:34.321645 9339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:34.323630 9339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:34.323869 9339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:34.324124 9339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:34.329097 9350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:34.329077 9345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:34.329157 9347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:34.329494 9339 server_base.cc:1034] running on GCE node
I20260807 06:33:34.329718 9339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:34.329957 9339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:34.331122 9339 hybrid_clock.cc:648] HybridClock initialized: now 1786084414331113 us; error 29 us; skew 500 ppm
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:34.333892 9339 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:34.334376 9339 webserver.cc:460] Webserver started at http://127.2.243.3:37411/ using document root <none> and password file <none>
I20260807 06:33:34.334649 9339 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:34.334733 9339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:34.334872 9339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:34.335914 9339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "f9d164473a6f42c5af51373a57c3434d"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "bdfd70f55b18fe7996021c70cdb87519"
server_key_iv: "f614831d96df15417b506ab027cac8fc"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.336289 9339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "f9d164473a6f42c5af51373a57c3434d"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "bdfd70f55b18fe7996021c70cdb87519"
server_key_iv: "f614831d96df15417b506ab027cac8fc"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.337776 9339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:34.338577 9363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.338778 9339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:34.338881 9339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "f9d164473a6f42c5af51373a57c3434d"
format_stamp: "Formatted at 2026-08-07 06:33:34 on dist-test-slave-rwrg"
server_key: "bdfd70f55b18fe7996021c70cdb87519"
server_key_iv: "f614831d96df15417b506ab027cac8fc"
server_key_version: "encryptionkey@0"
I20260807 06:33:34.338974 9339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:34.352463 9339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:34.354954 9339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:34.355136 9339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:34.355419 9339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:34.355749 9339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:34.355804 9339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.355845 9339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:34.355890 9339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:34.362205 9339 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:35119
I20260807 06:33:34.362262 9529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:35119 every 8 connection(s)
I20260807 06:33:34.362612 9339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:34.364661 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 9339
I20260807 06:33:34.364750 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:34.364921 3020 external_mini_cluster.cc:1428] Setting key 97d75adf7132d453bc28365ae7925f33
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:34.373026 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.363417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:43007 (local address 127.2.243.62:39793)
0807 06:33:34.363538 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:34.363541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:34.363710 (+ 169us) server_negotiation.cc:407] Connection header received
0807 06:33:34.364819 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:34.364820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:34.364834 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:34.364895 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:34.365792 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.366369 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.367446 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.367626 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.367678 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.370018 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:34.370020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:34.370021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:34.370029 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:34.371571 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.371997 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.371999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.371999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.372065 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.372335 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.372336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.372336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.372456 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:34.372570 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:34.372852 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 06:33:34.372932 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:33:34.373231 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.363374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39793 (local address 127.2.243.3:43007)
0807 06:33:34.363566 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:34.363617 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:34.364655 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:34.365002 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:34.365007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:34.365391 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:34.365690 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:34.365697 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.366456 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.366457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:34.367360 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:34.367362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.367704 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.367705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:34.367815 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.368277 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:34.368286 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:34.369913 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:34.371679 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:34.371687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:34.371689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:34.371917 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:34.372140 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:34.372141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:34.372141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:34.372256 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:34.372563 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:34.372578 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:34.372717 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 06:33:34.372829 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:33:34.372989 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260807 06:33:34.373746 9530 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39793
I20260807 06:33:34.373865 9530 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:34.374069 9530 heartbeater.cc:511] Master 127.2.243.62:39793 requested a full tablet report, sending...
I20260807 06:33:34.374466 8825 ts_manager.cc:194] Registered new tserver with Master: f9d164473a6f42c5af51373a57c3434d (127.2.243.3:35119)
I20260807 06:33:34.374830 8825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:43007
I20260807 06:33:34.376718 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:34.402230 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.394763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.2.243.1:35631)
0807 06:33:34.394949 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:34.394955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:34.394966 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:34.394993 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:34.394995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:34.395017 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:34.395088 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:34.395626 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.396342 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.397094 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.397413 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.397470 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.399671 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:34.399697 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:34.399697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:34.399704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:34.401158 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.401466 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.401469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.401469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.401529 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.401717 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.401717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.401718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.401843 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:34.401949 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:34.402027 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 06:33:34.402095 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[8733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:34.410091 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:34.403613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.2.243.62:39793)
0807 06:33:34.403670 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:34.403672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:34.403684 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:34.403729 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:34.403730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:34.403738 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:34.403783 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:34.404181 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.404707 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.405327 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:34.405501 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:34.405556 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:34.407724 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:34.407726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:34.407726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:34.407732 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:34.409065 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.409389 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.409390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.409390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.409437 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:34.409668 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:34.409668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:34.409668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:34.409798 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:34.409871 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:34.409959 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 06:33:34.410017 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 06:33:34.410343 8825 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43050
I20260807 06:33:34.410763 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 8933
I20260807 06:33:34.426710 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 9147
I20260807 06:33:34.436862 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 9339
I20260807 06:33:34.455142 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 8760
2026-08-07T06:33:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2098 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[9555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[9555](info): set up 2 sockets
Aug 07 06:33:34 dist-test-slave-rwrg krb5kdc[9555](info): commencing operation
krb5kdc: starting...
W20260807 06:33:35.754438 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:35 dist-test-slave-rwrg krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:35.879513 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45983
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:45983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:35.961954 9572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:35.962167 9572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:35.962190 9572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:35.963258 9572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:35.963290 9572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:35.963304 9572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:35.963317 9572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:35.963330 9572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:35.964795 9572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:45983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:35.964999 9572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:35.965163 9572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:33:35.970116 9572 server_base.cc:1034] running on GCE node
W20260807 06:33:35.970068 9579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:35.970072 9584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:35.970090 9580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:35.970588 9572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:35.970928 9572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:35.972102 9572 hybrid_clock.cc:648] HybridClock initialized: now 1786084415972082 us; error 37 us; skew 500 ppm
Aug 07 06:33:35 dist-test-slave-rwrg krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084415, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:35.974732 9572 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:35.975113 9572 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:35.975184 9572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:35.975322 9572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:35.976327 9572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "2925601aa2714837bc6c5fb96aa65076"
format_stamp: "Formatted at 2026-08-07 06:33:35 on dist-test-slave-rwrg"
server_key: "a459e7abdafc340254ede522bce56721"
server_key_iv: "f7ea5170356a5ff7e0daa9a9a4a41d17"
server_key_version: "encryptionkey@0"
I20260807 06:33:35.976734 9572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "2925601aa2714837bc6c5fb96aa65076"
format_stamp: "Formatted at 2026-08-07 06:33:35 on dist-test-slave-rwrg"
server_key: "a459e7abdafc340254ede522bce56721"
server_key_iv: "f7ea5170356a5ff7e0daa9a9a4a41d17"
server_key_version: "encryptionkey@0"
I20260807 06:33:35.978055 9572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:35.978811 9596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:35.978997 9572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:35.979110 9572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "2925601aa2714837bc6c5fb96aa65076"
format_stamp: "Formatted at 2026-08-07 06:33:35 on dist-test-slave-rwrg"
server_key: "a459e7abdafc340254ede522bce56721"
server_key_iv: "f7ea5170356a5ff7e0daa9a9a4a41d17"
server_key_version: "encryptionkey@0"
I20260807 06:33:35.979185 9572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:35.993422 9572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:35.995321 9572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:35.995494 9572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:35.999825 9572 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:45983
I20260807 06:33:35.999851 9678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:45983 every 8 connection(s)
I20260807 06:33:36.000157 9572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:36.001359 9679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:36.003654 9679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076: Bootstrap starting.
I20260807 06:33:36.004266 9679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:36.004680 9679 log.cc:824] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:36.004709 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 9572
I20260807 06:33:36.004812 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:36.005019 3020 external_mini_cluster.cc:1428] Setting key 8e73cd81f0d61e287ec7cf0896cf4d0b
I20260807 06:33:36.005548 9679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076: No bootstrap required, opened a new log
I20260807 06:33:36.007254 9679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2925601aa2714837bc6c5fb96aa65076" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45983 } }
I20260807 06:33:36.007418 9679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:36.007457 9679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2925601aa2714837bc6c5fb96aa65076, State: Initialized, Role: FOLLOWER
I20260807 06:33:36.007592 9679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2925601aa2714837bc6c5fb96aa65076" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45983 } }
I20260807 06:33:36.007682 9679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:36.007725 9679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:36.007781 9679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:36.008819 9679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2925601aa2714837bc6c5fb96aa65076" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45983 } }
I20260807 06:33:36.008910 9679 leader_election.cc:302] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2925601aa2714837bc6c5fb96aa65076; no voters:
I20260807 06:33:36.009114 9679 leader_election.cc:288] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:36.009164 9687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:36.009336 9687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [term 1 LEADER]: Becoming Leader. State: Replica: 2925601aa2714837bc6c5fb96aa65076, State: Running, Role: LEADER
I20260807 06:33:36.009410 9679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:36.009433 9687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2925601aa2714837bc6c5fb96aa65076" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45983 } }
I20260807 06:33:36.009775 9688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2925601aa2714837bc6c5fb96aa65076" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2925601aa2714837bc6c5fb96aa65076" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45983 } } }
I20260807 06:33:36.009874 9688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:36.009768 9687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2925601aa2714837bc6c5fb96aa65076. Latest consensus state: current_term: 1 leader_uuid: "2925601aa2714837bc6c5fb96aa65076" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2925601aa2714837bc6c5fb96aa65076" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45983 } } }
I20260807 06:33:36.010066 9687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:36.010203 9696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:36.010936 9696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:36.013083 9696 catalog_manager.cc:1347] Generated new cluster ID: 0ac45c239822464d827bb2cb26d49134
I20260807 06:33:36.013136 9696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:36 dist-test-slave-rwrg krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:36.018086 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.005591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.2.243.62:45983)
0807 06:33:36.005790 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.005799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.005836 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:33:36.007235 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.007244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.007296 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.007457 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:36.008262 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.009460 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.010659 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.011252 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.011378 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.014282 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:36.014286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:36.014286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:36.014298 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:36.016288 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.016731 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.016734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.016734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.016852 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.017103 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.017103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.017104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.017384 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:36.017526 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.017646 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.017879 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260807 06:33:36.024330 9696 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:36.024951 9696 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:36.028053 9696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076: Generated new TSK 0
I20260807 06:33:36.028250 9696 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:36.059252 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45983
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:36.141722 9724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:36.141970 9724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:36.142019 9724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:36.143219 9724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:36.143285 9724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:36.143334 9724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:36.144990 9724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:36.145246 9724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:36.145489 9724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:33:36.150775 9724 server_base.cc:1034] running on GCE node
W20260807 06:33:36.150746 9732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.150734 9736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.150745 9731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:36.151158 9724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:36.151364 9724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:36.152558 9724 hybrid_clock.cc:648] HybridClock initialized: now 1786084416152565 us; error 55 us; skew 500 ppm
Aug 07 06:33:36 dist-test-slave-rwrg krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:36.155241 9724 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:36.155547 9724 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:36.155592 9724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:36.155735 9724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:36.156886 9724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "46c2021c7dec4e079a9c6626b0073d49"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "deb211b6c4d73e0ce5d89a0acf39a6eb"
server_key_iv: "1296e8e599af837abfca98795d7a8979"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.157243 9724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "46c2021c7dec4e079a9c6626b0073d49"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "deb211b6c4d73e0ce5d89a0acf39a6eb"
server_key_iv: "1296e8e599af837abfca98795d7a8979"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.158632 9724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:36.159399 9744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.159749 9724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:36.159880 9724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "46c2021c7dec4e079a9c6626b0073d49"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "deb211b6c4d73e0ce5d89a0acf39a6eb"
server_key_iv: "1296e8e599af837abfca98795d7a8979"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.159962 9724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:36.179600 9724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:36.181770 9724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:36.181903 9724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:36.182224 9724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:36.182580 9724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:36.182641 9724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.182689 9724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:36.182742 9724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.189134 9724 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:37561
I20260807 06:33:36.189206 9925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:37561 every 8 connection(s)
I20260807 06:33:36.189530 9724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:36.194762 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 9724
I20260807 06:33:36.194850 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:36.195050 3020 external_mini_cluster.cc:1428] Setting key f4983b9ceefd1426cff2b020e5138cc1
Aug 07 06:33:36 dist-test-slave-rwrg krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:36.199703 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.190351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45983 (local address 127.2.243.1:60545)
0807 06:33:36.190535 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.190579 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.191075 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.191307 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.191311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.191582 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:36.192095 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.192100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.192884 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.192885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.193712 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.193715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.194074 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.194075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.194178 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.194679 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:36.194687 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:36.196300 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:36.198028 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:36.198036 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:36.198039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:36.198381 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:36.198666 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:36.198667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:36.198668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:36.198826 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:36.199188 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:36.199195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:36.199315 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.199406 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.199536 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":115,"threads_started":1}
I20260807 06:33:36.199769 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.190402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:60545 (local address 127.2.243.62:45983)
0807 06:33:36.190470 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.190512 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.190673 (+ 161us) server_negotiation.cc:407] Connection header received
0807 06:33:36.191179 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.191179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.191192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.191254 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:36.192210 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.192787 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.193800 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.193995 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.194048 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.196402 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:36.196403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:36.196404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:36.196412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:36.197939 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.198483 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.198485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.198485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.198581 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.198931 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.198933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.198933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.199098 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:36.199181 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.199467 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.199622 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:33:36.200407 9926 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45983
I20260807 06:33:36.200562 9926 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:36.200747 9926 heartbeater.cc:511] Master 127.2.243.62:45983 requested a full tablet report, sending...
I20260807 06:33:36.201290 9619 ts_manager.cc:194] Registered new tserver with Master: 46c2021c7dec4e079a9c6626b0073d49 (127.2.243.1:37561)
I20260807 06:33:36.201939 9619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:60545
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:36.224578 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45983
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:36.306581 9941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:36.306780 9941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:36.306803 9941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:36.307897 9941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:36.307937 9941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:36.308003 9941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:36.309605 9941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:36.309844 9941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:36.310075 9941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:36.315128 9952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.315133 9950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.315140 9949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:36.315378 9941 server_base.cc:1034] running on GCE node
I20260807 06:33:36.315605 9941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:36.315834 9941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:36.316995 9941 hybrid_clock.cc:648] HybridClock initialized: now 1786084416316983 us; error 42 us; skew 500 ppm
Aug 07 06:33:36 dist-test-slave-rwrg krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:36.319445 9941 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:36.319844 9941 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:36.319916 9941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:36.320106 9941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:36.321215 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "ced484acca224a2d95183b836cc4375e"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "e8bdf0bf2478b4b02e30f92717d2b59c"
server_key_iv: "b039334634254108c773e2d7bd8e74e2"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.321581 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "ced484acca224a2d95183b836cc4375e"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "e8bdf0bf2478b4b02e30f92717d2b59c"
server_key_iv: "b039334634254108c773e2d7bd8e74e2"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.322888 9941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:36.323613 9959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.323786 9941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:36.323887 9941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "ced484acca224a2d95183b836cc4375e"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "e8bdf0bf2478b4b02e30f92717d2b59c"
server_key_iv: "b039334634254108c773e2d7bd8e74e2"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.323963 9941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:36.338630 9941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:36.341105 9941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:36.341279 9941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:36.341511 9941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:36.341858 9941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:36.341914 9941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.341957 9941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:36.341984 9941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.347803 9941 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:46275
I20260807 06:33:36.347890 10137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:46275 every 8 connection(s)
I20260807 06:33:36.348182 9941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:36.349808 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 9941
I20260807 06:33:36.349895 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:36.350148 3020 external_mini_cluster.cc:1428] Setting key c297da950e529e9a041ad30d3df89fb6
Aug 07 06:33:36 dist-test-slave-rwrg krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:36.358867 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.349130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:59207 (local address 127.2.243.62:45983)
0807 06:33:36.349239 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.349242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.349536 (+ 294us) server_negotiation.cc:407] Connection header received
0807 06:33:36.350074 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.350076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.350091 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.350173 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:36.351297 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.352045 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.353568 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.353750 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.353810 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.356072 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:36.356074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:36.356074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:36.356082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:36.357495 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.357929 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.357930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.357930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.357988 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.358238 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.358239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.358239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.358354 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:36.358433 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.358698 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.358777 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:33:36.359076 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.349102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45983 (local address 127.2.243.2:59207)
0807 06:33:36.349358 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.349396 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.349967 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.350276 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.350279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.350635 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:36.351201 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.351206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.352138 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.352139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.353437 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.353440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.353833 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.353834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.353970 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.354643 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:36.354651 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:36.355970 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:36.357615 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:36.357622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:36.357624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:36.357837 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:36.358056 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:36.358057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:36.358057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:36.358168 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:36.358425 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:36.358431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:36.358570 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.358679 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.358832 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":160,"threads_started":1}
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
I20260807 06:33:36.359607 10138 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45983
I20260807 06:33:36.359714 10138 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:36.359874 10138 heartbeater.cc:511] Master 127.2.243.62:45983 requested a full tablet report, sending...
I20260807 06:33:36.360204 9619 ts_manager.cc:194] Registered new tserver with Master: ced484acca224a2d95183b836cc4375e (127.2.243.2:46275)
I20260807 06:33:36.360658 9619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:59207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:36.380304 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45983
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:43283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:36.463202 10143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:36.463479 10143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:36.463507 10143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:36.464605 10143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:36.464639 10143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:36.464663 10143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:36.466204 10143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:43283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:36.466389 10143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:36.466557 10143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:36.471486 10152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.471501 10154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.471527 10150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:36.471781 10143 server_base.cc:1034] running on GCE node
I20260807 06:33:36.472074 10143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:36.472304 10143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:36.473469 10143 hybrid_clock.cc:648] HybridClock initialized: now 1786084416473425 us; error 75 us; skew 500 ppm
Aug 07 06:33:36 dist-test-slave-rwrg krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:36.476017 10143 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:36.476446 10143 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:36.476630 10143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:36.476815 10143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:36.478018 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "a47d68f79e0446edaaa7773fd94c6590"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "f76940b5ca244d04003f8fbfe5ff1007"
server_key_iv: "336c856be2054f7b39a24e95852980d0"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.478430 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "a47d68f79e0446edaaa7773fd94c6590"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "f76940b5ca244d04003f8fbfe5ff1007"
server_key_iv: "336c856be2054f7b39a24e95852980d0"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.479916 10143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:36.480712 10160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.480887 10143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:36.480993 10143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "a47d68f79e0446edaaa7773fd94c6590"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "f76940b5ca244d04003f8fbfe5ff1007"
server_key_iv: "336c856be2054f7b39a24e95852980d0"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.481073 10143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:36.495332 10143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:36.497990 10143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:36.498195 10143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:36.498472 10143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:36.498893 10143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:36.498956 10143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.499011 10143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:36.499069 10143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.506333 10143 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:45335
I20260807 06:33:36.506385 10330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:45335 every 8 connection(s)
I20260807 06:33:36.506693 10143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:33:36 dist-test-slave-rwrg krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:36.516014 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.507454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:39857 (local address 127.2.243.62:45983)
0807 06:33:36.507543 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.507546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.507745 (+ 199us) server_negotiation.cc:407] Connection header received
0807 06:33:36.508202 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.508203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.508214 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.508290 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:36.509322 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.509816 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.510901 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.511149 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.511244 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.513356 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:36.513358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:36.513358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:36.513365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:36.514796 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.515170 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.515171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.515171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.515232 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.515479 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.515479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.515479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.515596 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:36.515654 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.515882 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.515949 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:33:36.516203 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.507410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45983 (local address 127.2.243.3:39857)
0807 06:33:36.507623 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.507659 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.508110 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.508337 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.508341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.508663 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:36.509204 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.509210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.509975 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.509976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.510808 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.510810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.511249 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.511250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.511347 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.511857 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:36.511865 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:36.513267 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:36.514870 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:36.514875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:36.514877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:36.515083 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:36.515315 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:36.515315 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:36.515316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:36.515411 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:36.515657 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:36.515661 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:36.515769 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.515850 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.515964 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":123,"threads_started":1}
I20260807 06:33:36.516563 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 10143
I20260807 06:33:36.516688 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:36.516801 10331 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45983
I20260807 06:33:36.516897 10331 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:36.516912 3020 external_mini_cluster.cc:1428] Setting key dd436a9fe00e672e2a15a595cfd53a2d
I20260807 06:33:36.517052 10331 heartbeater.cc:511] Master 127.2.243.62:45983 requested a full tablet report, sending...
I20260807 06:33:36.517424 9619 ts_manager.cc:194] Registered new tserver with Master: a47d68f79e0446edaaa7773fd94c6590 (127.2.243.3:45335)
I20260807 06:33:36.517931 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:33:36.517939 9619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:39857
I20260807 06:33:36.525421 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.519353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.2.243.62:45983)
0807 06:33:36.519393 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.519395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.519502 (+ 107us) server_negotiation.cc:407] Connection header received
0807 06:33:36.519559 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.519560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.519567 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.519611 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:36.520138 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.520711 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.521377 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.521588 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.521643 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.522956 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:36.522957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:36.522957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:36.522964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:36.524369 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.524742 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.524742 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.524743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.524856 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:36.525022 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:36.525022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:36.525022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:36.525142 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:36.525213 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.525306 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.525362 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:33:36.527155 9619 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:36.528975 9619 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:36.535804 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.531404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45335 (local address 127.0.0.1:59376)
0807 06:33:36.531512 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.531564 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.531630 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.532020 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.532024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.532027 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:36.532097 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.532098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.533515 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.533515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.534732 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.534735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.535526 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.535527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.535592 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.535598 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.535671 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.535736 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":26,"thread_start_us":56,"threads_started":1}
I20260807 06:33:36.535861 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.531592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41250 (local address 127.2.243.1:37561)
0807 06:33:36.532230 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.532234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.532243 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:36.532281 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.532283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.532297 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.532368 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:36.532595 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.533598 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.534653 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.535345 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.535425 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.535489 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.535730 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.535770 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":85,"threads_started":1}
I20260807 06:33:36.536144 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.531731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.2.243.3:45335)
0807 06:33:36.531872 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.531876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.531884 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:36.531911 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.531913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.531924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.532014 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:36.532305 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.533394 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.534817 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.535448 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.535513 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.535560 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.536045 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.536084 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260807 06:33:36.536343 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.531418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:37561 (local address 127.0.0.1:41250)
0807 06:33:36.531554 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.531564 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.531630 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.532451 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.532452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.532454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:36.532487 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.532488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.533691 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.533693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.534558 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.534560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.535440 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.535441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.535573 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.535596 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.535678 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.536277 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":548,"thread_start_us":81,"threads_started":1}
I20260807 06:33:36.536742 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.531603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:46275 (local address 127.0.0.1:40832)
0807 06:33:36.531968 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.532055 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.532109 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.532354 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.532355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.532358 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:36.532410 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.532412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.533370 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.533371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.534732 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.534735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.536487 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.536488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.536578 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.536583 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.536647 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.536684 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":70,"thread_start_us":65,"threads_started":1}
I20260807 06:33:36.536789 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.531968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.2.243.2:46275)
0807 06:33:36.532174 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.532179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.532189 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:36.532229 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.532231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.532247 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.532521 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:36.532602 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.533264 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.535594 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.536396 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.536480 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.536562 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.536660 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.536705 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
I20260807 06:33:36.536880 10234 tablet_service.cc:1467] Processing CreateTablet for tablet 2617797cb7b74378ba8e6491448e1c20 (DEFAULT_TABLE table=test-table [id=469dd12c9e23426a96df444bb1ee94cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:36.537067 9819 tablet_service.cc:1467] Processing CreateTablet for tablet 2617797cb7b74378ba8e6491448e1c20 (DEFAULT_TABLE table=test-table [id=469dd12c9e23426a96df444bb1ee94cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:36.537279 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2617797cb7b74378ba8e6491448e1c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:36.537364 9819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2617797cb7b74378ba8e6491448e1c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:36.537416 10034 tablet_service.cc:1467] Processing CreateTablet for tablet 2617797cb7b74378ba8e6491448e1c20 (DEFAULT_TABLE table=test-table [id=469dd12c9e23426a96df444bb1ee94cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:36.537712 10034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2617797cb7b74378ba8e6491448e1c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:36.539860 10359 tablet_bootstrap.cc:492] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49: Bootstrap starting.
I20260807 06:33:36.540581 10359 tablet_bootstrap.cc:654] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:36.540805 10360 tablet_bootstrap.cc:492] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Bootstrap starting.
I20260807 06:33:36.540870 10363 tablet_bootstrap.cc:492] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Bootstrap starting.
I20260807 06:33:36.541281 10359 log.cc:824] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:36.541483 10363 tablet_bootstrap.cc:654] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:36.541656 10360 tablet_bootstrap.cc:654] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:36.542092 10363 log.cc:824] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:36.542272 10359 tablet_bootstrap.cc:492] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49: No bootstrap required, opened a new log
I20260807 06:33:36.542263 10360 log.cc:824] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:36.542336 10359 ts_tablet_manager.cc:1397] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 06:33:36.542848 10363 tablet_bootstrap.cc:492] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: No bootstrap required, opened a new log
I20260807 06:33:36.542930 10363 ts_tablet_manager.cc:1397] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:36.544091 10359 raft_consensus.cc:348] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.544224 10359 raft_consensus.cc:374] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:36.544246 10359 raft_consensus.cc:729] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c2021c7dec4e079a9c6626b0073d49, State: Initialized, Role: FOLLOWER
I20260807 06:33:36.544283 10363 raft_consensus.cc:348] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.544373 10359 consensus_queue.cc:260] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.544481 10363 raft_consensus.cc:374] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:36.544559 10363 raft_consensus.cc:729] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ced484acca224a2d95183b836cc4375e, State: Initialized, Role: FOLLOWER
I20260807 06:33:36.544672 10363 consensus_queue.cc:260] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.544739 10359 ts_tablet_manager.cc:1428] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 06:33:36.544754 9926 heartbeater.cc:503] Master 127.2.243.62:45983 was elected leader, sending a full tablet report...
I20260807 06:33:36.544951 10363 ts_tablet_manager.cc:1428] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:33:36.544987 10138 heartbeater.cc:503] Master 127.2.243.62:45983 was elected leader, sending a full tablet report...
I20260807 06:33:36.545547 10360 tablet_bootstrap.cc:492] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: No bootstrap required, opened a new log
I20260807 06:33:36.545603 10360 ts_tablet_manager.cc:1397] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 06:33:36.547370 10360 raft_consensus.cc:348] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.547492 10360 raft_consensus.cc:374] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:36.547521 10360 raft_consensus.cc:729] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47d68f79e0446edaaa7773fd94c6590, State: Initialized, Role: FOLLOWER
I20260807 06:33:36.547662 10360 consensus_queue.cc:260] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.547917 10360 ts_tablet_manager.cc:1428] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:33:36.547961 10331 heartbeater.cc:503] Master 127.2.243.62:45983 was elected leader, sending a full tablet report...
I20260807 06:33:36.579516 10370 raft_consensus.cc:482] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:36.579728 10370 raft_consensus.cc:504] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.580132 10370 leader_election.cc:288] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a47d68f79e0446edaaa7773fd94c6590 (127.2.243.3:45335), ced484acca224a2d95183b836cc4375e (127.2.243.2:46275)
I20260807 06:33:36.583941 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.580474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:47221 (local address 127.2.243.3:45335)
0807 06:33:36.580598 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.580601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.580611 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:36.580640 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.580641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.580655 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.580708 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:36.580897 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.581704 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.583140 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.583578 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.583663 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.583673 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.583837 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.583873 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 06:33:36.583941 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.580360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:45335 (local address 127.2.243.1:47221)
0807 06:33:36.580422 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.580436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.580505 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.580729 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.580735 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.580738 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:36.580827 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.580829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.581782 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.581782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.582999 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.583002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.583662 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.583663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.583759 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.583762 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.583829 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.583870 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 06:33:36.584190 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.580478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:46275 (local address 127.2.243.1:46695)
0807 06:33:36.580936 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.580946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.581022 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.581159 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.581160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.581161 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:36.581252 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.581253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.582100 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.582101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.583208 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:36.583210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.583872 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.583874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:36.583964 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.583967 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.584041 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.584087 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":391,"threads_started":1}
I20260807 06:33:36.584291 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.580641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:46695 (local address 127.2.243.2:46275)
0807 06:33:36.580688 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.580690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.581035 (+ 345us) server_negotiation.cc:407] Connection header received
0807 06:33:36.581066 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.581067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.581081 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.581141 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:36.581354 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.582018 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.583319 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.583789 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.583871 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.583880 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.584174 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.584220 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:33:36.584285 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2617797cb7b74378ba8e6491448e1c20" candidate_uuid: "46c2021c7dec4e079a9c6626b0073d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47d68f79e0446edaaa7773fd94c6590" is_pre_election: true
I20260807 06:33:36.584452 10278 raft_consensus.cc:2454] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c2021c7dec4e079a9c6626b0073d49 in term 0.
I20260807 06:33:36.584633 10064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2617797cb7b74378ba8e6491448e1c20" candidate_uuid: "46c2021c7dec4e079a9c6626b0073d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced484acca224a2d95183b836cc4375e" is_pre_election: true
I20260807 06:33:36.584704 9752 leader_election.cc:302] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c2021c7dec4e079a9c6626b0073d49, a47d68f79e0446edaaa7773fd94c6590; no voters:
I20260807 06:33:36.584815 10064 raft_consensus.cc:2454] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46c2021c7dec4e079a9c6626b0073d49 in term 0.
I20260807 06:33:36.584893 10370 raft_consensus.cc:2781] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:36.584992 10370 raft_consensus.cc:482] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:36.585021 10370 raft_consensus.cc:3037] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:36.585712 10370 raft_consensus.cc:504] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.585844 10370 leader_election.cc:288] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [CANDIDATE]: Term 1 election: Requested vote from peers a47d68f79e0446edaaa7773fd94c6590 (127.2.243.3:45335), ced484acca224a2d95183b836cc4375e (127.2.243.2:46275)
I20260807 06:33:36.586040 10064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2617797cb7b74378ba8e6491448e1c20" candidate_uuid: "46c2021c7dec4e079a9c6626b0073d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced484acca224a2d95183b836cc4375e"
I20260807 06:33:36.586040 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2617797cb7b74378ba8e6491448e1c20" candidate_uuid: "46c2021c7dec4e079a9c6626b0073d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47d68f79e0446edaaa7773fd94c6590"
I20260807 06:33:36.586118 10064 raft_consensus.cc:3037] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:36.586118 10278 raft_consensus.cc:3037] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:36.587020 10064 raft_consensus.cc:2454] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c2021c7dec4e079a9c6626b0073d49 in term 1.
I20260807 06:33:36.587074 10278 raft_consensus.cc:2454] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46c2021c7dec4e079a9c6626b0073d49 in term 1.
I20260807 06:33:36.587198 9752 leader_election.cc:302] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46c2021c7dec4e079a9c6626b0073d49, a47d68f79e0446edaaa7773fd94c6590; no voters:
I20260807 06:33:36.587308 10370 raft_consensus.cc:2781] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:36.587456 10370 raft_consensus.cc:686] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [term 1 LEADER]: Becoming Leader. State: Replica: 46c2021c7dec4e079a9c6626b0073d49, State: Running, Role: LEADER
I20260807 06:33:36.587551 10370 consensus_queue.cc:237] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } }
I20260807 06:33:36.588495 9617 catalog_manager.cc:5495] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46c2021c7dec4e079a9c6626b0073d49 (127.2.243.1). New cstate: current_term: 1 leader_uuid: "46c2021c7dec4e079a9c6626b0073d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46c2021c7dec4e079a9c6626b0073d49" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 37561 } health_report { overall_health: HEALTHY } } }
I20260807 06:33:36.597146 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.594928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.2.243.1:37561)
0807 06:33:36.594969 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.594971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.594981 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:36.595062 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.595063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.595077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.595138 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:36.595258 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.595693 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.596415 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:36.596673 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:36.596731 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:36.596882 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:36.596971 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:36.597034 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.597068 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.597083 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 06:33:36.598790 10139 tablet.cc:2369] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:36.598973 10064 raft_consensus.cc:1261] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 1 FOLLOWER]: Refusing update from remote peer 46c2021c7dec4e079a9c6626b0073d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:36.599007 10278 raft_consensus.cc:1261] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 1 FOLLOWER]: Refusing update from remote peer 46c2021c7dec4e079a9c6626b0073d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:36.599363 10377 consensus_queue.cc:1035] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ced484acca224a2d95183b836cc4375e" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 46275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:36.599472 10370 consensus_queue.cc:1035] T 2617797cb7b74378ba8e6491448e1c20 P 46c2021c7dec4e079a9c6626b0073d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a47d68f79e0446edaaa7773fd94c6590" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 45335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:36.601739 10381 mvcc.cc:205] Tried to move back new op lower bound from 7315801770386558976 to 7315801770343346176. Current Snapshot: MvccSnapshot[applied={T|T < 7315801770386558976}]
I20260807 06:33:36.601917 10384 mvcc.cc:205] Tried to move back new op lower bound from 7315801770386558976 to 7315801770343346176. Current Snapshot: MvccSnapshot[applied={T|T < 7315801770386558976}]
I20260807 06:33:36.602473 10382 mvcc.cc:205] Tried to move back new op lower bound from 7315801770386558976 to 7315801770343346176. Current Snapshot: MvccSnapshot[applied={T|T < 7315801770386558976}]
I20260807 06:33:36.603873 9616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:36.603952 9616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:36.604693 9616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2925601aa2714837bc6c5fb96aa65076: Sending DeleteTablet for 3 replicas of tablet 2617797cb7b74378ba8e6491448e1c20
I20260807 06:33:36.605016 10234 tablet_service.cc:1514] Processing DeleteTablet for tablet 2617797cb7b74378ba8e6491448e1c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:36 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:59376
I20260807 06:33:36.605031 10034 tablet_service.cc:1514] Processing DeleteTablet for tablet 2617797cb7b74378ba8e6491448e1c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:36 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:40832
I20260807 06:33:36.605247 10396 tablet_replica.cc:330] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: stopping tablet replica
I20260807 06:33:36.605245 9819 tablet_service.cc:1514] Processing DeleteTablet for tablet 2617797cb7b74378ba8e6491448e1c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:36 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:41250
I20260807 06:33:36.605269 10398 tablet_replica.cc:330] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: stopping tablet replica
I20260807 06:33:36.605365 10396 raft_consensus.cc:2229] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:36.605465 10396 raft_consensus.cc:2258] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:36.605482 10398 raft_consensus.cc:2229] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:36.605629 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 9724
I20260807 06:33:36.605997 10396 ts_tablet_manager.cc:1905] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:36.608206 10398 raft_consensus.cc:2258] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:36.610659 10398 ts_tablet_manager.cc:1905] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:36.612269 10398 ts_tablet_manager.cc:1918] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:36.612341 10398 log.cc:1196] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/2617797cb7b74378ba8e6491448e1c20
I20260807 06:33:36.612643 10398 ts_tablet_manager.cc:1939] T 2617797cb7b74378ba8e6491448e1c20 P ced484acca224a2d95183b836cc4375e: Deleting consensus metadata
I20260807 06:33:36.613092 9601 catalog_manager.cc:4841] TS ced484acca224a2d95183b836cc4375e (127.2.243.2:46275): tablet 2617797cb7b74378ba8e6491448e1c20 (table test-table [id=469dd12c9e23426a96df444bb1ee94cd]) successfully deleted
I20260807 06:33:36.614349 10396 ts_tablet_manager.cc:1918] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:36.614447 10396 log.cc:1196] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/2617797cb7b74378ba8e6491448e1c20
I20260807 06:33:36.614722 10396 ts_tablet_manager.cc:1939] T 2617797cb7b74378ba8e6491448e1c20 P a47d68f79e0446edaaa7773fd94c6590: Deleting consensus metadata
I20260807 06:33:36.615155 9601 catalog_manager.cc:4841] TS a47d68f79e0446edaaa7773fd94c6590 (127.2.243.3:45335): tablet 2617797cb7b74378ba8e6491448e1c20 (table test-table [id=469dd12c9e23426a96df444bb1ee94cd]) successfully deleted
W20260807 06:33:36.620900 9600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:37561 (error 108)
I20260807 06:33:36.621201 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 9941
I20260807 06:33:36.621483 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.621328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:37561 (local address 127.0.0.1:41270)
0807 06:33:36.621381 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.621440 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:37561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 06:33:36.629014 9600 catalog_manager.cc:4584] TS 46c2021c7dec4e079a9c6626b0073d49 (127.2.243.1:37561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2617797cb7b74378ba8e6491448e1c20: Network error: Client connection negotiation failed: client connection to 127.2.243.1:37561: connect: Connection refused (error 111)
I20260807 06:33:36.637214 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 10143
I20260807 06:33:36.651016 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 9572
2026-08-07T06:33:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2195 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T06:33:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:36Z Disabled control of system clock
I20260807 06:33:36.685173 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43975
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:43975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:33:36.766836 10405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:36.767081 10405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:36.767119 10405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:36.768203 10405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:36.768270 10405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:36.768312 10405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:36.768345 10405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:36.768378 10405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:36.769809 10405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:43975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:36.770092 10405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:36.770376 10405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:36.775336 10415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.775378 10412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.775389 10411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:36.775468 10405 server_base.cc:1034] running on GCE node
I20260807 06:33:36.775838 10405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:36.776135 10405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:36.777324 10405 hybrid_clock.cc:648] HybridClock initialized: now 1786084416777301 us; error 41 us; skew 500 ppm
I20260807 06:33:36.778554 10405 webserver.cc:460] Webserver started at http://127.2.243.62:39729/ using document root <none> and password file <none>
I20260807 06:33:36.778795 10405 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:36.778851 10405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:36.779031 10405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:36.780114 10405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "d788a260ccea48eda617abcc51f850f8"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "01bddb14011d55f631d8089cf89d3569"
server_key_iv: "feaf3f5e4fe1864c2311c7dec651a6ec"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.780467 10405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "d788a260ccea48eda617abcc51f850f8"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "01bddb14011d55f631d8089cf89d3569"
server_key_iv: "feaf3f5e4fe1864c2311c7dec651a6ec"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.781811 10405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:36.782567 10428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.782785 10405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:36.782886 10405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "d788a260ccea48eda617abcc51f850f8"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "01bddb14011d55f631d8089cf89d3569"
server_key_iv: "feaf3f5e4fe1864c2311c7dec651a6ec"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.782958 10405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:36.786391 10405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:36.786646 10405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:36.786788 10405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:36.790742 10405 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:43975
I20260807 06:33:36.790794 10511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:43975 every 8 connection(s)
I20260807 06:33:36.791114 10405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:36.791786 10512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:36.794704 10512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8: Bootstrap starting.
I20260807 06:33:36.795316 10512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:36.795621 10512 log.cc:824] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:36.796327 10512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8: No bootstrap required, opened a new log
I20260807 06:33:36.797953 10512 raft_consensus.cc:348] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d788a260ccea48eda617abcc51f850f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43975 } }
I20260807 06:33:36.798101 10512 raft_consensus.cc:374] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:36.798125 10512 raft_consensus.cc:729] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d788a260ccea48eda617abcc51f850f8, State: Initialized, Role: FOLLOWER
I20260807 06:33:36.798285 10512 consensus_queue.cc:260] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d788a260ccea48eda617abcc51f850f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43975 } }
I20260807 06:33:36.798350 10512 raft_consensus.cc:388] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:36.798372 10512 raft_consensus.cc:482] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:36.798426 10512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:36.798983 10512 raft_consensus.cc:504] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d788a260ccea48eda617abcc51f850f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43975 } }
I20260807 06:33:36.799098 10512 leader_election.cc:302] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d788a260ccea48eda617abcc51f850f8; no voters:
I20260807 06:33:36.799289 10512 leader_election.cc:288] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:36.799342 10519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:36.799576 10512 sys_catalog.cc:564] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:36.799634 10519 raft_consensus.cc:686] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [term 1 LEADER]: Becoming Leader. State: Replica: d788a260ccea48eda617abcc51f850f8, State: Running, Role: LEADER
I20260807 06:33:36.799731 10519 consensus_queue.cc:237] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d788a260ccea48eda617abcc51f850f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43975 } }
I20260807 06:33:36.799823 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 10405
I20260807 06:33:36.799912 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:36.800105 3020 external_mini_cluster.cc:1428] Setting key 2b97f13e2b377fdc1bf222b6d2b71f43
I20260807 06:33:36.800086 10519 sys_catalog.cc:455] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d788a260ccea48eda617abcc51f850f8. Latest consensus state: current_term: 1 leader_uuid: "d788a260ccea48eda617abcc51f850f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d788a260ccea48eda617abcc51f850f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43975 } } }
I20260807 06:33:36.800149 10519 sys_catalog.cc:458] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:36.800521 10520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d788a260ccea48eda617abcc51f850f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d788a260ccea48eda617abcc51f850f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43975 } } }
I20260807 06:33:36.800607 10520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:36.800755 10534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:36.801316 10534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:36.803009 10534 catalog_manager.cc:1347] Generated new cluster ID: 8c39f88c2aa545ee8e16b7136b69c547
I20260807 06:33:36.803062 10534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:36.803464 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.800846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.2.243.62:43975)
0807 06:33:36.801655 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.801665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.801699 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:33:36.802014 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.802024 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.802077 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.802233 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:36.802929 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:36.802932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:36.802933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:36.802936 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:36.802974 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:36.802989 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:36.803053 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.803239 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.803297 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":59,"threads_started":1}
I20260807 06:33:36.817145 10534 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:36.817308 10534 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:36.822803 10534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8: Generated new TSK 0
I20260807 06:33:36.823602 10534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:33:36.827062 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43975
--builtin_ntp_servers=127.2.243.20:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:33:36.910207 10554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:36.910434 10554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:36.910456 10554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:36.911634 10554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:36.911675 10554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:36.911741 10554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:36.913175 10554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:36.913447 10554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:36.913769 10554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:36.919548 10563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.919616 10560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:36.919625 10565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:36.919891 10554 server_base.cc:1034] running on GCE node
I20260807 06:33:36.920228 10554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:36.920522 10554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:36.921695 10554 hybrid_clock.cc:648] HybridClock initialized: now 1786084416921673 us; error 44 us; skew 500 ppm
I20260807 06:33:36.922991 10554 webserver.cc:460] Webserver started at http://127.2.243.1:42837/ using document root <none> and password file <none>
I20260807 06:33:36.923287 10554 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:36.923365 10554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:36.923520 10554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:36.924717 10554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "a94bc717276e4df2856da1ddec444cb3"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "350bfe305db2f4c92c3e181672de3661"
server_key_iv: "0632eb44e7e1a366215eae01140d8886"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.925078 10554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "a94bc717276e4df2856da1ddec444cb3"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "350bfe305db2f4c92c3e181672de3661"
server_key_iv: "0632eb44e7e1a366215eae01140d8886"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.926383 10554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:36.927090 10575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.927263 10554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:36.927369 10554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "a94bc717276e4df2856da1ddec444cb3"
format_stamp: "Formatted at 2026-08-07 06:33:36 on dist-test-slave-rwrg"
server_key: "350bfe305db2f4c92c3e181672de3661"
server_key_iv: "0632eb44e7e1a366215eae01140d8886"
server_key_version: "encryptionkey@0"
I20260807 06:33:36.927443 10554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:36.930882 10554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:36.931159 10554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:36.931303 10554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:36.931509 10554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:36.931823 10554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:36.931878 10554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.931919 10554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:36.931947 10554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:36.937863 10554 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:42365
I20260807 06:33:36.937938 10748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:42365 every 8 connection(s)
I20260807 06:33:36.938274 10554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:36.941423 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.939065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:38899 (local address 127.2.243.62:43975)
0807 06:33:36.939132 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:36.939135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:36.939414 (+ 279us) server_negotiation.cc:407] Connection header received
0807 06:33:36.939860 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:36.939861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:36.939868 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:36.939923 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:36.941233 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:36.941234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:36.941234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:36.941238 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:36.941252 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:36.941256 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:36.941306 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:36.941355 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 06:33:36.941369 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:33:36.941630 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:36.939056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43975 (local address 127.2.243.1:38899)
0807 06:33:36.939288 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:36.939327 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:36.939774 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:36.939975 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:36.939978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:36.939986 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:36.941093 (+ 1107us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:36.941094 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:36.941122 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:36.941151 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:36.941303 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:36.941305 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:36.941307 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:36.941357 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:33:36.941403 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":107,"threads_started":1}
I20260807 06:33:36.942214 10749 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43975
I20260807 06:33:36.942315 10749 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:36.942335 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 10554
I20260807 06:33:36.942421 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:36.942469 10749 heartbeater.cc:511] Master 127.2.243.62:43975 requested a full tablet report, sending...
I20260807 06:33:36.942618 3020 external_mini_cluster.cc:1428] Setting key 1f21d41a7798dee30614323c58f41c4b
I20260807 06:33:36.943029 10451 ts_manager.cc:194] Registered new tserver with Master: a94bc717276e4df2856da1ddec444cb3 (127.2.243.1:42365)
I20260807 06:33:36.943714 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43975
--builtin_ntp_servers=127.2.243.20:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:33:37.024937 10752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:37.025202 10752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:37.025231 10752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:37.026471 10752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:37.026517 10752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:37.026544 10752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:37.027850 10752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:37.028048 10752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:37.028295 10752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:37.033540 10759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.033540 10768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.033742 10763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:37.033690 10752 server_base.cc:1034] running on GCE node
I20260807 06:33:37.034032 10752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:37.034245 10752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:37.035386 10752 hybrid_clock.cc:648] HybridClock initialized: now 1786084417035362 us; error 44 us; skew 500 ppm
I20260807 06:33:37.036764 10752 webserver.cc:460] Webserver started at http://127.2.243.2:45949/ using document root <none> and password file <none>
I20260807 06:33:37.037024 10752 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:37.037070 10752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:37.037283 10752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:37.038415 10752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "2d76b4c750984ff1982c9a9721f60233"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "3c2e0537af31091fc3da8c216f29cafd"
server_key_iv: "e09ad3fc1610d88d4bc8bd29981cde27"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.038738 10752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "2d76b4c750984ff1982c9a9721f60233"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "3c2e0537af31091fc3da8c216f29cafd"
server_key_iv: "e09ad3fc1610d88d4bc8bd29981cde27"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.040040 10752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:37.040894 10779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.041079 10752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:33:37.041185 10752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "2d76b4c750984ff1982c9a9721f60233"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "3c2e0537af31091fc3da8c216f29cafd"
server_key_iv: "e09ad3fc1610d88d4bc8bd29981cde27"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.041273 10752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:37.044656 10752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:37.044927 10752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:37.045070 10752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:37.045274 10752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:37.045598 10752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:37.045655 10752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.045696 10752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:37.045723 10752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.051635 10752 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:40507
I20260807 06:33:37.051676 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:40507 every 8 connection(s)
I20260807 06:33:37.052035 10752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:37.054931 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.052807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:47551 (local address 127.2.243.62:43975)
0807 06:33:37.052854 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.052856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.053092 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:33:37.053522 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.053523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.053530 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.053588 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.054697 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.054699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.054699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.054703 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.054720 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.054725 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.054795 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.054862 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.054875 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:37.055114 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.052804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43975 (local address 127.2.243.2:47551)
0807 06:33:37.052971 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.053004 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.053437 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.053641 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.053644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.053652 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:37.054526 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:37.054527 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:37.054555 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:37.054589 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:37.054794 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:37.054797 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:37.054799 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:37.054849 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:33:37.054895 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":91,"threads_started":1}
I20260807 06:33:37.055601 10970 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43975
I20260807 06:33:37.055696 10970 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:37.055807 10970 heartbeater.cc:511] Master 127.2.243.62:43975 requested a full tablet report, sending...
I20260807 06:33:37.056190 10451 ts_manager.cc:194] Registered new tserver with Master: 2d76b4c750984ff1982c9a9721f60233 (127.2.243.2:40507)
I20260807 06:33:37.058838 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 10752
I20260807 06:33:37.058923 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:37.059092 3020 external_mini_cluster.cc:1428] Setting key 16042f1d851b2335e9f0a60b4503e0d7
I20260807 06:33:37.060061 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43975
--builtin_ntp_servers=127.2.243.20:42859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:33:37.142863 10979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:37.143070 10979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:37.143092 10979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:37.144289 10979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:37.144326 10979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:37.144351 10979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:37.145810 10979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:37.145995 10979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:37.146227 10979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:37.151366 10987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.151378 10990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.151391 10986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:37.151719 10979 server_base.cc:1034] running on GCE node
I20260807 06:33:37.151952 10979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:37.152189 10979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:37.153347 10979 hybrid_clock.cc:648] HybridClock initialized: now 1786084417153322 us; error 43 us; skew 500 ppm
I20260807 06:33:37.154611 10979 webserver.cc:460] Webserver started at http://127.2.243.3:42527/ using document root <none> and password file <none>
I20260807 06:33:37.154860 10979 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:37.154928 10979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:37.155103 10979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:37.156186 10979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "c7ca9acc224243f883c6110a3557614d"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "5df13aaa4ae536596e12e907daa633cd"
server_key_iv: "40b87b3dbbebbcd4dc21e143d4e0ec8d"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.156620 10979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "c7ca9acc224243f883c6110a3557614d"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "5df13aaa4ae536596e12e907daa633cd"
server_key_iv: "40b87b3dbbebbcd4dc21e143d4e0ec8d"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.157899 10979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:37.158619 11000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.158790 10979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:37.158890 10979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "c7ca9acc224243f883c6110a3557614d"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "5df13aaa4ae536596e12e907daa633cd"
server_key_iv: "40b87b3dbbebbcd4dc21e143d4e0ec8d"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.158974 10979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:37.162497 10979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:37.162768 10979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:37.162914 10979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:37.163131 10979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:37.163458 10979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:37.163520 10979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.163568 10979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:37.163595 10979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.169196 10979 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:46681
I20260807 06:33:37.169313 11185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:46681 every 8 connection(s)
I20260807 06:33:37.169641 10979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:37.172654 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.170431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:46029 (local address 127.2.243.62:43975)
0807 06:33:37.170677 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.170679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.170752 (+ 73us) server_negotiation.cc:407] Connection header received
0807 06:33:37.171213 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.171214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.171218 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.171288 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.172390 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.172391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.172392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.172395 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.172405 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.172408 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.172452 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.172589 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.172602 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:33:37.172870 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.170397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43975 (local address 127.2.243.3:46029)
0807 06:33:37.170635 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.170685 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.171124 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.171340 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.171342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.171350 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:37.172262 (+ 912us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:37.172262 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:37.172290 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:37.172317 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:37.172452 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:37.172454 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:37.172456 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:37.172608 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 06:33:37.172660 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260807 06:33:37.173336 11187 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43975
I20260807 06:33:37.173425 11187 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:37.173547 11187 heartbeater.cc:511] Master 127.2.243.62:43975 requested a full tablet report, sending...
I20260807 06:33:37.173892 10451 ts_manager.cc:194] Registered new tserver with Master: c7ca9acc224243f883c6110a3557614d (127.2.243.3:46681)
I20260807 06:33:37.175489 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 10979
I20260807 06:33:37.175618 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:37.175817 3020 external_mini_cluster.cc:1428] Setting key 77db108060cf1c734438c32df08c19e7
I20260807 06:33:37.176842 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:33:37.179219 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.178169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.2.243.62:43975)
0807 06:33:37.178234 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.178236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.178371 (+ 135us) server_negotiation.cc:407] Connection header received
0807 06:33:37.178420 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.178421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.178425 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.178475 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.178994 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.178995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.178995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.178999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.179015 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.179019 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.179116 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.179158 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.179171 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:33:37.181142 10451 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:37.182780 10451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:37.187069 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.185230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:40507 (local address 127.0.0.1:48272)
0807 06:33:37.185604 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.185645 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.185771 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.185995 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.186001 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.186008 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:37.186524 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:37.186537 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:37.186554 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:37.186581 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:37.186874 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:37.186881 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:37.186884 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:37.186952 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 06:33:37.186965 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":69,"threads_started":1}
I20260807 06:33:37.187101 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.185339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.2.243.2:40507)
0807 06:33:37.185753 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.185758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.185773 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:33:37.185867 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.185871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.185883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.185989 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.186703 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.186713 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.186713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.186718 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.186755 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.186783 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.186824 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.186972 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.186992 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":89,"threads_started":1}
I20260807 06:33:37.187393 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.185677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53974 (local address 127.2.243.3:46681)
0807 06:33:37.185834 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.185839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.185953 (+ 114us) server_negotiation.cc:407] Connection header received
0807 06:33:37.185992 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.185994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.186004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.186079 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.186905 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.186911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.186911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.186916 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.186953 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.186987 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.187048 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.187298 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.187314 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260807 06:33:37.187230 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.185438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:46681 (local address 127.0.0.1:53974)
0807 06:33:37.185828 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.185839 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.185887 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.186373 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.186374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.186377 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:37.186796 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:37.186797 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:37.186805 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:37.186824 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:37.187078 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:37.187079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:37.187080 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:33:37.187132 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:33:37.187146 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":90,"threads_started":1}
I20260807 06:33:37.187587 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.185298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:42365 (local address 127.0.0.1:48158)
0807 06:33:37.185602 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.185643 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.185699 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.186786 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.186787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.186790 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:37.187181 (+ 391us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:37.187181 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:37.187189 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:37.187203 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:37.187448 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:37.187449 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:37.187450 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:33:37.187479 (+ 29us) client_negotiation.cc:240] Negotiation successful
0807 06:33:37.187494 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":120,"threads_started":1}
I20260807 06:33:37.187884 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.185416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48158 (local address 127.2.243.1:42365)
0807 06:33:37.186384 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.186389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.186401 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:37.186428 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.186430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.186440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.186558 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.187297 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.187304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.187304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.187308 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.187357 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.187379 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.187760 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.187784 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.187800 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":924,"thread_start_us":107,"threads_started":1}
I20260807 06:33:37.188328 10865 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf4131e708f48d894c09488b4be676e (DEFAULT_TABLE table=test-table [id=739c682d21a14a21895044e3ee3dff35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:37.188479 10646 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf4131e708f48d894c09488b4be676e (DEFAULT_TABLE table=test-table [id=739c682d21a14a21895044e3ee3dff35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:37.188735 10865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf4131e708f48d894c09488b4be676e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:37.188865 10646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf4131e708f48d894c09488b4be676e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:37.188923 11081 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf4131e708f48d894c09488b4be676e (DEFAULT_TABLE table=test-table [id=739c682d21a14a21895044e3ee3dff35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:37.189291 11081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf4131e708f48d894c09488b4be676e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:37.191478 11207 tablet_bootstrap.cc:492] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3: Bootstrap starting.
I20260807 06:33:37.192077 11207 tablet_bootstrap.cc:654] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:37.192384 11207 log.cc:824] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:37.192457 11209 tablet_bootstrap.cc:492] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Bootstrap starting.
I20260807 06:33:37.192768 11211 tablet_bootstrap.cc:492] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Bootstrap starting.
I20260807 06:33:37.193168 11209 tablet_bootstrap.cc:654] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:37.193482 11209 log.cc:824] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:37.193512 11207 tablet_bootstrap.cc:492] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3: No bootstrap required, opened a new log
I20260807 06:33:37.193537 11211 tablet_bootstrap.cc:654] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:37.193609 11207 ts_tablet_manager.cc:1397] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:33:37.193827 11211 log.cc:824] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:37.194243 11209 tablet_bootstrap.cc:492] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: No bootstrap required, opened a new log
I20260807 06:33:37.194391 11209 ts_tablet_manager.cc:1397] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:37.194624 11211 tablet_bootstrap.cc:492] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: No bootstrap required, opened a new log
I20260807 06:33:37.194689 11211 ts_tablet_manager.cc:1397] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:37.195755 11207 raft_consensus.cc:348] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.195905 11207 raft_consensus.cc:374] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:37.195943 11207 raft_consensus.cc:729] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94bc717276e4df2856da1ddec444cb3, State: Initialized, Role: FOLLOWER
I20260807 06:33:37.196077 11207 consensus_queue.cc:260] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.196377 10749 heartbeater.cc:503] Master 127.2.243.62:43975 was elected leader, sending a full tablet report...
I20260807 06:33:37.196362 11207 ts_tablet_manager.cc:1428] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:33:37.197378 11211 raft_consensus.cc:348] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.197541 11211 raft_consensus.cc:374] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:37.197603 11211 raft_consensus.cc:729] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ca9acc224243f883c6110a3557614d, State: Initialized, Role: FOLLOWER
I20260807 06:33:37.197556 11209 raft_consensus.cc:348] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.197695 11209 raft_consensus.cc:374] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:37.197738 11209 raft_consensus.cc:729] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d76b4c750984ff1982c9a9721f60233, State: Initialized, Role: FOLLOWER
I20260807 06:33:37.197741 11211 consensus_queue.cc:260] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.197857 11209 consensus_queue.cc:260] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.198040 11211 ts_tablet_manager.cc:1428] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 06:33:37.198099 11187 heartbeater.cc:503] Master 127.2.243.62:43975 was elected leader, sending a full tablet report...
I20260807 06:33:37.198139 11209 ts_tablet_manager.cc:1428] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260807 06:33:37.198278 10970 heartbeater.cc:503] Master 127.2.243.62:43975 was elected leader, sending a full tablet report...
I20260807 06:33:37.285907 11217 raft_consensus.cc:482] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:37.286059 11217 raft_consensus.cc:504] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.286468 11217 leader_election.cc:288] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a94bc717276e4df2856da1ddec444cb3 (127.2.243.1:42365), 2d76b4c750984ff1982c9a9721f60233 (127.2.243.2:40507)
I20260807 06:33:37.295254 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.286774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:41325 (local address 127.2.243.2:40507)
0807 06:33:37.286820 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.286822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.286832 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:37.286862 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.286863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.286870 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.286931 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.294954 (+ 8023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.294956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.294956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.294961 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.294989 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.294994 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.295088 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.295136 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.295152 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:33:37.295298 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.286745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:35625 (local address 127.2.243.1:42365)
0807 06:33:37.286793 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.286795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.286982 (+ 187us) server_negotiation.cc:407] Connection header received
0807 06:33:37.287014 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.287015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.287022 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.287099 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.294983 (+ 7884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.294984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.294985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.294989 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.295009 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.295013 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.295093 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.295210 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.295224 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:33:37.295291 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.286685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:40507 (local address 127.2.243.3:41325)
0807 06:33:37.286737 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.286750 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.286796 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.286969 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.286982 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.286984 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:37.294831 (+ 7847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:37.294832 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:37.294843 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:37.294856 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:37.295079 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:37.295080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:37.295081 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:33:37.295132 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:33:37.295146 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 06:33:37.295437 11219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.286686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:42365 (local address 127.2.243.3:35625)
0807 06:33:37.286879 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.286889 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.286945 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.287180 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.287181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.287185 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:37.294831 (+ 7646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:33:37.294831 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:37.294840 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:33:37.294856 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:37.295118 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:37.295119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:37.295120 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 06:33:37.295207 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:33:37.295221 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":12,"thread_start_us":124,"threads_started":1}
I20260807 06:33:37.295753 10892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf4131e708f48d894c09488b4be676e" candidate_uuid: "c7ca9acc224243f883c6110a3557614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d76b4c750984ff1982c9a9721f60233" is_pre_election: true
I20260807 06:33:37.295851 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf4131e708f48d894c09488b4be676e" candidate_uuid: "c7ca9acc224243f883c6110a3557614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a94bc717276e4df2856da1ddec444cb3" is_pre_election: true
I20260807 06:33:37.295976 10680 raft_consensus.cc:2454] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ca9acc224243f883c6110a3557614d in term 0.
I20260807 06:33:37.295955 10892 raft_consensus.cc:2454] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ca9acc224243f883c6110a3557614d in term 0.
I20260807 06:33:37.296255 11003 leader_election.cc:302] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a94bc717276e4df2856da1ddec444cb3, c7ca9acc224243f883c6110a3557614d; no voters:
I20260807 06:33:37.296468 11217 raft_consensus.cc:2781] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:37.296584 11217 raft_consensus.cc:482] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:37.296615 11217 raft_consensus.cc:3037] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:37.297358 11217 raft_consensus.cc:504] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.297518 11217 leader_election.cc:288] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [CANDIDATE]: Term 1 election: Requested vote from peers a94bc717276e4df2856da1ddec444cb3 (127.2.243.1:42365), 2d76b4c750984ff1982c9a9721f60233 (127.2.243.2:40507)
I20260807 06:33:37.297677 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf4131e708f48d894c09488b4be676e" candidate_uuid: "c7ca9acc224243f883c6110a3557614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a94bc717276e4df2856da1ddec444cb3"
I20260807 06:33:37.297788 10680 raft_consensus.cc:3037] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:37.297782 10892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cf4131e708f48d894c09488b4be676e" candidate_uuid: "c7ca9acc224243f883c6110a3557614d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d76b4c750984ff1982c9a9721f60233"
I20260807 06:33:37.297916 10892 raft_consensus.cc:3037] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:37.298621 10680 raft_consensus.cc:2454] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ca9acc224243f883c6110a3557614d in term 1.
I20260807 06:33:37.298749 10892 raft_consensus.cc:2454] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ca9acc224243f883c6110a3557614d in term 1.
I20260807 06:33:37.298830 11003 leader_election.cc:302] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a94bc717276e4df2856da1ddec444cb3, c7ca9acc224243f883c6110a3557614d; no voters:
I20260807 06:33:37.298928 11217 raft_consensus.cc:2781] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:37.299220 11217 raft_consensus.cc:686] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 1 LEADER]: Becoming Leader. State: Replica: c7ca9acc224243f883c6110a3557614d, State: Running, Role: LEADER
I20260807 06:33:37.299351 11217 consensus_queue.cc:237] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } }
I20260807 06:33:37.300266 10451 catalog_manager.cc:5495] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d reported cstate change: term changed from 0 to 1, leader changed from <none> to c7ca9acc224243f883c6110a3557614d (127.2.243.3). New cstate: current_term: 1 leader_uuid: "c7ca9acc224243f883c6110a3557614d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7ca9acc224243f883c6110a3557614d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 46681 } health_report { overall_health: HEALTHY } } }
W20260807 06:33:37.302558 10973 tablet.cc:2369] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:37.332998 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.331832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.2.243.3:46681)
0807 06:33:37.331891 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.331894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.331902 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:37.331946 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.331946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.331959 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.332011 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.332719 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.332721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.332722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.332727 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.332748 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.332753 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.332815 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.332902 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.332917 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:33:37.335307 10680 raft_consensus.cc:1261] T 9cf4131e708f48d894c09488b4be676e P a94bc717276e4df2856da1ddec444cb3 [term 1 FOLLOWER]: Refusing update from remote peer c7ca9acc224243f883c6110a3557614d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:37.335386 10892 raft_consensus.cc:1261] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 1 FOLLOWER]: Refusing update from remote peer c7ca9acc224243f883c6110a3557614d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:37.335731 11217 consensus_queue.cc:1035] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d76b4c750984ff1982c9a9721f60233" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:37.335855 11220 consensus_queue.cc:1035] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a94bc717276e4df2856da1ddec444cb3" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 42365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:37.338340 11225 mvcc.cc:205] Tried to move back new op lower bound from 7315801773402525696 to 7315801773258854400. Current Snapshot: MvccSnapshot[applied={T|T < 7315801773402525696}]
I20260807 06:33:37.341796 10449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:37.341912 10449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:37.342720 10449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d788a260ccea48eda617abcc51f850f8: Sending DeleteTablet for 3 replicas of tablet 9cf4131e708f48d894c09488b4be676e
I20260807 06:33:37.343200 10865 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf4131e708f48d894c09488b4be676e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:37 UTC) from {username='slave'} at 127.0.0.1:48272
I20260807 06:33:37.343331 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 10554
I20260807 06:33:37.343436 11081 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cf4131e708f48d894c09488b4be676e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:37 UTC) from {username='slave'} at 127.0.0.1:53974
I20260807 06:33:37.346933 11242 tablet_replica.cc:330] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: stopping tablet replica
I20260807 06:33:37.347064 11242 raft_consensus.cc:2229] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:37.347177 11242 raft_consensus.cc:2258] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:37.347882 11242 ts_tablet_manager.cc:1905] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:37.349654 11242 ts_tablet_manager.cc:1918] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:37.349721 11242 log.cc:1196] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/9cf4131e708f48d894c09488b4be676e
I20260807 06:33:37.349958 11242 ts_tablet_manager.cc:1939] T 9cf4131e708f48d894c09488b4be676e P 2d76b4c750984ff1982c9a9721f60233: Deleting consensus metadata
I20260807 06:33:37.350543 10431 catalog_manager.cc:4841] TS 2d76b4c750984ff1982c9a9721f60233 (127.2.243.2:40507): tablet 9cf4131e708f48d894c09488b4be676e (table test-table [id=739c682d21a14a21895044e3ee3dff35]) successfully deleted
I20260807 06:33:37.355024 11243 tablet_replica.cc:330] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: stopping tablet replica
I20260807 06:33:37.355176 11243 raft_consensus.cc:2229] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:37.355366 11243 raft_consensus.cc:2258] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 06:33:37.355890 10432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:42365 (error 108)
I20260807 06:33:37.355942 11243 ts_tablet_manager.cc:1905] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:37.356534 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 10752
I20260807 06:33:37.356763 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.356627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:42365 (local address 127.0.0.1:48164)
0807 06:33:37.356679 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.356721 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:33:37.356871 10432 catalog_manager.cc:4584] TS a94bc717276e4df2856da1ddec444cb3 (127.2.243.1:42365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cf4131e708f48d894c09488b4be676e: Network error: Client connection negotiation failed: client connection to 127.2.243.1:42365: connect: Connection refused (error 111)
I20260807 06:33:37.359817 11243 ts_tablet_manager.cc:1918] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:37.359894 11243 log.cc:1196] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/9cf4131e708f48d894c09488b4be676e
I20260807 06:33:37.360180 11243 ts_tablet_manager.cc:1939] T 9cf4131e708f48d894c09488b4be676e P c7ca9acc224243f883c6110a3557614d: Deleting consensus metadata
I20260807 06:33:37.360757 10431 catalog_manager.cc:4841] TS c7ca9acc224243f883c6110a3557614d (127.2.243.3:46681): tablet 9cf4131e708f48d894c09488b4be676e (table test-table [id=739c682d21a14a21895044e3ee3dff35]) successfully deleted
I20260807 06:33:37.366561 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 10979
I20260807 06:33:37.381557 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 10405
2026-08-07T06:33:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (727 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 06:33:37.404875 3020 mini_oidc.cc:152] Starting JWKS server
I20260807 06:33:37.406134 3020 webserver.cc:460] Webserver started at https://127.0.0.1:46075/ using document root <none> and password file <none>
I20260807 06:33:37.406219 3020 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:33:37.406418 3020 webserver.cc:460] Webserver started at http://127.0.0.1:44609/ using document root <none> and password file <none>
2026-08-07T06:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:37Z Disabled control of system clock
I20260807 06:33:37.415355 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:34715
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:34715
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46075/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/testchainca.cert with env {}
W20260807 06:33:37.495034 11251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:37.495230 11251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:37.495251 11251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:37.496541 11251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:37.496582 11251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:37.496610 11251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:37.496623 11251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:37.496670 11251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:33:37.496687 11251 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:33:37.496708 11251 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46075/jwks.json
I20260807 06:33:37.498286 11251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:34715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:34715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46075/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:37.498560 11251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:37.498786 11251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:37.503762 11266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.503806 11261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:37.503778 11251 server_base.cc:1034] running on GCE node
W20260807 06:33:37.503834 11264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:37.504235 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:37.504469 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:37.505694 11251 hybrid_clock.cc:648] HybridClock initialized: now 1786084417505673 us; error 40 us; skew 500 ppm
I20260807 06:33:37.506983 11251 webserver.cc:460] Webserver started at http://127.2.243.62:44533/ using document root <none> and password file <none>
I20260807 06:33:37.507272 11251 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:37.507352 11251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:37.507489 11251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:37.508634 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "79e39d5b791257f23d2a94703a8a8f90"
server_key_iv: "613df4d9debd958acebdc605c10dc118"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.508982 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "79e39d5b791257f23d2a94703a8a8f90"
server_key_iv: "613df4d9debd958acebdc605c10dc118"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.510295 11251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:37.511056 11272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.511274 11251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:37.511368 11251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "79e39d5b791257f23d2a94703a8a8f90"
server_key_iv: "613df4d9debd958acebdc605c10dc118"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.511422 11251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:37.526998 11251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:37.527266 11251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:37.527379 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:37.531795 11251 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:34715
I20260807 06:33:37.531827 11354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:34715 every 8 connection(s)
I20260807 06:33:37.532169 11251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:37.533238 11355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:37.535528 11355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4: Bootstrap starting.
I20260807 06:33:37.536108 11355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:37.536398 11355 log.cc:824] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:37.537194 11355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4: No bootstrap required, opened a new log
I20260807 06:33:37.538758 11355 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34715 } }
I20260807 06:33:37.538887 11355 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:37.538909 11355 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5f5cf2a56a45fbb7d3dbeb4851f6d4, State: Initialized, Role: FOLLOWER
I20260807 06:33:37.539095 11355 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34715 } }
I20260807 06:33:37.539175 11355 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:37.539214 11355 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:37.539263 11355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:37.539839 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 11251
I20260807 06:33:37.539891 11355 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34715 } }
I20260807 06:33:37.539947 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:37.540005 11355 leader_election.cc:302] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf5f5cf2a56a45fbb7d3dbeb4851f6d4; no voters:
I20260807 06:33:37.540163 3020 external_mini_cluster.cc:1428] Setting key 53c9b77153387dd81700be5a10a0a5ba
I20260807 06:33:37.540180 11355 leader_election.cc:288] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:37.540258 11360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:37.540410 11360 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [term 1 LEADER]: Becoming Leader. State: Replica: bf5f5cf2a56a45fbb7d3dbeb4851f6d4, State: Running, Role: LEADER
I20260807 06:33:37.540450 11355 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:37.540526 11360 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34715 } }
I20260807 06:33:37.540860 11360 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf5f5cf2a56a45fbb7d3dbeb4851f6d4. Latest consensus state: current_term: 1 leader_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34715 } } }
I20260807 06:33:37.540948 11360 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:37.541157 11362 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5f5cf2a56a45fbb7d3dbeb4851f6d4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34715 } } }
I20260807 06:33:37.541165 11373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:37.541828 11373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:37.542289 11362 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:37.543205 11373 catalog_manager.cc:1347] Generated new cluster ID: 2e85ba47f3764de7a8abfe673d3abdce
I20260807 06:33:37.543259 11373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:37.547780 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.540653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56632 (local address 127.2.243.62:34715)
0807 06:33:37.542389 (+ 1736us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.542399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.542436 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:33:37.542585 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.542590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.542642 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.542788 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.543036 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.544371 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.545510 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.546156 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.546286 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:37.546967 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.546979 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.546980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.547184 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.547225 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.547240 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.547313 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.547395 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.547576 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260807 06:33:37.548731 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:33:37.550522 11373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:37.551174 11373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:37.558581 11373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf5f5cf2a56a45fbb7d3dbeb4851f6d4: Generated new TSK 0
I20260807 06:33:37.558759 11373 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 06:33:37.585258 11404 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 06:33:37.585503 11404 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 06:33:37.585708 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.570481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.2.243.62:34715)
0807 06:33:37.570530 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.570532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.570653 (+ 121us) server_negotiation.cc:407] Connection header received
0807 06:33:37.570778 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.570780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.570792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.570859 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:33:37.571012 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.571498 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.572271 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.572475 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.572600 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:37.572838 (+ 238us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:33:37.585366 (+ 12528us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 06:33:37.585635 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 06:33:37.585743 11366 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 06:33:37.586092 11403 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:37.570412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:34715 (local address 127.0.0.1:56638)
0807 06:33:37.570578 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.570593 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.570675 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.570866 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.570867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.570869 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:33:37.570933 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:37.570938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.571591 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.571592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:37.572180 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:37.572182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.572604 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.572605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:37.572691 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:37.572710 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:33:37.585620 (+ 12910us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 06:33:37.585751 (+ 131us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.243.62:34715: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.243.62:34715: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 06:33:37.586562 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 11251
2026-08-07T06:33:37Z chronyd exiting
I20260807 06:33:37.628597 3020 test_util.cc:175] -----------------------------------------------
I20260807 06:33:37.628688 3020 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786084392481039-3020-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 06:33:37.629617 3020 mini_oidc.cc:152] Starting JWKS server
I20260807 06:33:37.630362 3020 webserver.cc:460] Webserver started at https://127.0.0.1:37019/ using document root <none> and password file <none>
I20260807 06:33:37.630407 3020 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:33:37.630607 3020 webserver.cc:460] Webserver started at http://127.0.0.1:33725/ using document root <none> and password file <none>
2026-08-07T06:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:37Z Disabled control of system clock
I20260807 06:33:37.639585 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42065
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42065
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37019/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/testchainca.cert with env {}
W20260807 06:33:37.724359 11411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:37.724638 11411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:37.724678 11411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:37.725811 11411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:37.725895 11411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:37.725919 11411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:37.725947 11411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:37.725973 11411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:33:37.726001 11411 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:33:37.726027 11411 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37019/jwks.json
I20260807 06:33:37.727571 11411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37019/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:37.727845 11411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:37.728066 11411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:37.733103 11416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.733179 11418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.733119 11421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:37.733320 11411 server_base.cc:1034] running on GCE node
I20260807 06:33:37.733590 11411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:37.733836 11411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:37.734978 11411 hybrid_clock.cc:648] HybridClock initialized: now 1786084417734962 us; error 47 us; skew 500 ppm
I20260807 06:33:37.736182 11411 webserver.cc:460] Webserver started at http://127.2.243.62:40979/ using document root <none> and password file <none>
I20260807 06:33:37.736440 11411 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:37.736485 11411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:37.736691 11411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:37.737815 11411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "2e0e02f20198431396ba1a970e1c248e"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "a05aefa95db12dca6cd0d7c74a1e5830"
server_key_iv: "713f1548c2df07f0d79704f2bf43374a"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.738190 11411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "2e0e02f20198431396ba1a970e1c248e"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "a05aefa95db12dca6cd0d7c74a1e5830"
server_key_iv: "713f1548c2df07f0d79704f2bf43374a"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.739496 11411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:37.740262 11430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.740444 11411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:37.740589 11411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "2e0e02f20198431396ba1a970e1c248e"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "a05aefa95db12dca6cd0d7c74a1e5830"
server_key_iv: "713f1548c2df07f0d79704f2bf43374a"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.740682 11411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:37.747752 11411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:37.748029 11411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:37.748173 11411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:37.752344 11411 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42065
I20260807 06:33:37.752375 11517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42065 every 8 connection(s)
I20260807 06:33:37.752760 11411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:37.753846 11518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:37.754184 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 11411
I20260807 06:33:37.754269 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:37.754472 3020 external_mini_cluster.cc:1428] Setting key 8a70c583779b07e046fafded6034721a
I20260807 06:33:37.756645 11518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e: Bootstrap starting.
I20260807 06:33:37.757413 11518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:37.757712 11518 log.cc:824] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:37.758453 11518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e: No bootstrap required, opened a new log
I20260807 06:33:37.760021 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.754949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.2.243.62:42065)
0807 06:33:37.755127 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.755134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.755156 (+ 22us) server_negotiation.cc:407] Connection header received
0807 06:33:37.755544 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.755549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.755635 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.755743 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:37.755981 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.756918 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.758060 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.758509 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.758727 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:37.759395 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:37.759399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:37.759399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:37.759565 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:37.759596 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:37.759606 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:37.759677 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:37.759769 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:33:37.759847 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260807 06:33:37.760560 11518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0e02f20198431396ba1a970e1c248e" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42065 } }
I20260807 06:33:37.760735 11518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:37.760776 11518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0e02f20198431396ba1a970e1c248e, State: Initialized, Role: FOLLOWER
I20260807 06:33:37.760900 11518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0e02f20198431396ba1a970e1c248e" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42065 } }
I20260807 06:33:37.760993 11518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:37.761032 11518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:37.761080 11518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:37.761164 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:33:37.761690 11518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0e02f20198431396ba1a970e1c248e" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42065 } }
I20260807 06:33:37.761818 11518 leader_election.cc:302] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e0e02f20198431396ba1a970e1c248e; no voters:
I20260807 06:33:37.762001 11518 leader_election.cc:288] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:37.762030 11530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:37.762145 11530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [term 1 LEADER]: Becoming Leader. State: Replica: 2e0e02f20198431396ba1a970e1c248e, State: Running, Role: LEADER
I20260807 06:33:37.762228 11530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0e02f20198431396ba1a970e1c248e" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42065 } }
I20260807 06:33:37.762288 11518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:37.762552 11532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e0e02f20198431396ba1a970e1c248e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0e02f20198431396ba1a970e1c248e" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42065 } } }
I20260807 06:33:37.762586 11534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e0e02f20198431396ba1a970e1c248e. Latest consensus state: current_term: 1 leader_uuid: "2e0e02f20198431396ba1a970e1c248e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0e02f20198431396ba1a970e1c248e" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42065 } } }
I20260807 06:33:37.762651 11532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:37.762660 11534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:37.762921 11540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:37.763501 11540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:37.765139 11540 catalog_manager.cc:1347] Generated new cluster ID: a81f566452b04a1693d6af83f9cac9dc
I20260807 06:33:37.765198 11540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:37.773051 11540 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:37.773579 11540 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:37.779359 11540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e0e02f20198431396ba1a970e1c248e: Generated new TSK 0
I20260807 06:33:37.779578 11540 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 06:33:37.805845 11566 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 06:33:37.805943 11566 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 06:33:37.806257 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:37.791770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.2.243.62:42065)
0807 06:33:37.791830 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:37.791833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:37.791937 (+ 104us) server_negotiation.cc:407] Connection header received
0807 06:33:37.791980 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:37.791981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:37.791994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:37.792075 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:33:37.792173 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.793001 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.793769 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.794080 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.794152 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:37.794324 (+ 172us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:33:37.806030 (+ 11706us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 06:33:37.806183 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 06:33:37.806316 11524 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 06:33:37.806389 11565 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:37.791655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42065 (local address 127.0.0.1:34128)
0807 06:33:37.791858 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:37.791869 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:37.791933 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:37.792079 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:37.792079 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:37.792081 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:33:37.792129 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:37.792131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.793075 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.793076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:37.793682 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:37.793684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:37.794158 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:37.794159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:37.794222 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:37.794227 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:33:37.806143 (+ 11916us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 06:33:37.806273 (+ 130us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.243.62:42065: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":129,"threads_started":1}
I20260807 06:33:37.806803 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 11411
2026-08-07T06:33:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 06:33:37.852241 3020 mini_oidc.cc:152] Starting JWKS server
I20260807 06:33:37.852982 3020 webserver.cc:460] Webserver started at https://127.0.0.1:46593/ using document root <none> and password file <none>
I20260807 06:33:37.853026 3020 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:33:37.853142 3020 webserver.cc:460] Webserver started at http://127.0.0.1:38405/ using document root <none> and password file <none>
2026-08-07T06:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:37Z Disabled control of system clock
I20260807 06:33:37.862049 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39529
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39529
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46593/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 06:33:37.943763 11573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:37.943964 11573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:37.943991 11573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:37.945153 11573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:37.945194 11573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:37.945207 11573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:37.945220 11573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:37.945232 11573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:33:37.945245 11573 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:33:37.945256 11573 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46593/jwks.json
I20260807 06:33:37.946622 11573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46593/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:37.946811 11573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:37.947026 11573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:37.951897 11580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.951920 11583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:37.951946 11581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:37.952148 11573 server_base.cc:1034] running on GCE node
I20260807 06:33:37.952414 11573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:37.952766 11573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:37.953919 11573 hybrid_clock.cc:648] HybridClock initialized: now 1786084417953910 us; error 46 us; skew 500 ppm
I20260807 06:33:37.955240 11573 webserver.cc:460] Webserver started at http://127.2.243.62:45625/ using document root <none> and password file <none>
I20260807 06:33:37.955492 11573 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:37.955561 11573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:37.955695 11573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:37.956872 11573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "1f1069b7eaea5d270c255e9698a99c45"
server_key_iv: "49005f2f22e84d435346c0dedb7a02df"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.957240 11573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "1f1069b7eaea5d270c255e9698a99c45"
server_key_iv: "49005f2f22e84d435346c0dedb7a02df"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.958571 11573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:37.959280 11595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:37.959455 11573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:37.959565 11573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c"
format_stamp: "Formatted at 2026-08-07 06:33:37 on dist-test-slave-rwrg"
server_key: "1f1069b7eaea5d270c255e9698a99c45"
server_key_iv: "49005f2f22e84d435346c0dedb7a02df"
server_key_version: "encryptionkey@0"
I20260807 06:33:37.959641 11573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:37.992280 11573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:37.992650 11573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:37.992815 11573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:37.997184 11573 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39529
I20260807 06:33:37.997262 11675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39529 every 8 connection(s)
I20260807 06:33:37.997634 11573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:37.998401 11678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:38.001365 11678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c: Bootstrap starting.
I20260807 06:33:38.001973 11678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:38.002278 11678 log.cc:824] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:38.002964 11678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c: No bootstrap required, opened a new log
I20260807 06:33:38.004549 11678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39529 } }
I20260807 06:33:38.004684 11678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:38.004706 11678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b9b5b52c7b401bbbf5f4fbd5ea598c, State: Initialized, Role: FOLLOWER
I20260807 06:33:38.004885 11678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39529 } }
I20260807 06:33:38.004948 11678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:38.004994 11678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:38.005045 11678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:38.005580 11678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39529 } }
I20260807 06:33:38.005693 11678 leader_election.cc:302] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76b9b5b52c7b401bbbf5f4fbd5ea598c; no voters:
I20260807 06:33:38.005883 11678 leader_election.cc:288] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:38.005932 11682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:38.006107 11682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [term 1 LEADER]: Becoming Leader. State: Replica: 76b9b5b52c7b401bbbf5f4fbd5ea598c, State: Running, Role: LEADER
I20260807 06:33:38.006187 11678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:38.006202 11682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39529 } }
I20260807 06:33:38.006641 11686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76b9b5b52c7b401bbbf5f4fbd5ea598c. Latest consensus state: current_term: 1 leader_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39529 } } }
I20260807 06:33:38.006727 11686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:38.006808 11684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b9b5b52c7b401bbbf5f4fbd5ea598c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39529 } } }
I20260807 06:33:38.006901 11684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:38.006897 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 11573
I20260807 06:33:38.006949 11690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:38.007017 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:38.007223 3020 external_mini_cluster.cc:1428] Setting key 353a439dc0c0770d260f74bcb283b66f
I20260807 06:33:38.007493 11690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:38.009265 11690 catalog_manager.cc:1347] Generated new cluster ID: 797512ce4f1248a18d73ac9a4d187026
I20260807 06:33:38.009321 11690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:38.013418 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:38.007701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.2.243.62:39529)
0807 06:33:38.007956 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:38.007967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:38.009043 (+ 1076us) server_negotiation.cc:407] Connection header received
0807 06:33:38.009177 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:38.009184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:38.009269 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:38.009430 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:38.009727 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.010617 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.011413 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.011837 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.011971 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:38.012713 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:38.012717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:38.012718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:38.012898 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:38.012931 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:38.012941 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:38.013028 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:38.013099 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 06:33:38.013264 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260807 06:33:38.014362 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:33:38.016760 11690 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:38.017241 11690 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:38.025604 11690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76b9b5b52c7b401bbbf5f4fbd5ea598c: Generated new TSK 0
I20260807 06:33:38.025820 11690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:33:38.040518 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:38.036771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36976 (local address 127.2.243.62:39529)
0807 06:33:38.036819 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:38.036821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:38.036926 (+ 105us) server_negotiation.cc:407] Connection header received
0807 06:33:38.036986 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:38.036987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:38.036999 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:38.037081 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:33:38.037193 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.037985 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.038732 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.039045 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.039119 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:38.039279 (+ 160us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:33:38.040255 (+ 976us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 06:33:38.040387 (+ 132us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 06:33:38.040647 11719 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:38.036703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39529 (local address 127.0.0.1:36976)
0807 06:33:38.036849 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:38.036861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:38.036929 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:38.037086 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:38.037086 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:38.037088 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:33:38.037138 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:38.037139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.038070 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.038071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:38.038654 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:38.038656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.039129 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.039129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:38.039173 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:38.039178 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:33:38.040358 (+ 1180us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 06:33:38.040466 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.243.62:39529: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260807 06:33:38.040855 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 11573
E20260807 06:33:38.065158 11720 webserver.cc:533] Webserver: sslize failed:
E20260807 06:33:38.065236 11720 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T06:33:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (244 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T06:33:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:38Z Disabled control of system clock
I20260807 06:33:38.104815 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39165
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:36981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39165
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/testchainca.cert with env {}
W20260807 06:33:38.186769 11728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:38.186969 11728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:38.186990 11728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:38.188118 11728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:38.188161 11728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:38.188174 11728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:38.188185 11728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:38.188197 11728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:33:38.188208 11728 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:33:38.188274 11728 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 06:33:38.189862 11728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39165
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:38.190109 11728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:38.190335 11728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:38.195149 11741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:38.195155 11736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:38.195228 11735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:38.195461 11728 server_base.cc:1034] running on GCE node
I20260807 06:33:38.195694 11728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:38.195994 11728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:38.197151 11728 hybrid_clock.cc:648] HybridClock initialized: now 1786084418197132 us; error 42 us; skew 500 ppm
I20260807 06:33:38.198812 11728 webserver.cc:460] Webserver started at http://127.2.243.62:43161/ using document root <none> and password file <none>
I20260807 06:33:38.199060 11728 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:38.199120 11728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:38.199302 11728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:38.200397 11728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "6383e61222ae4a35bcd3f8fa6ca4e507"
format_stamp: "Formatted at 2026-08-07 06:33:38 on dist-test-slave-rwrg"
server_key: "4eab7cafd8fd9cb2eb3259ba3a8126f9"
server_key_iv: "2b2a12ff4262a5904e8657d3971d24ec"
server_key_version: "encryptionkey@0"
I20260807 06:33:38.200874 11728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "6383e61222ae4a35bcd3f8fa6ca4e507"
format_stamp: "Formatted at 2026-08-07 06:33:38 on dist-test-slave-rwrg"
server_key: "4eab7cafd8fd9cb2eb3259ba3a8126f9"
server_key_iv: "2b2a12ff4262a5904e8657d3971d24ec"
server_key_version: "encryptionkey@0"
I20260807 06:33:38.202236 11728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:38.203018 11751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:38.203310 11728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:38.203430 11728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "6383e61222ae4a35bcd3f8fa6ca4e507"
format_stamp: "Formatted at 2026-08-07 06:33:38 on dist-test-slave-rwrg"
server_key: "4eab7cafd8fd9cb2eb3259ba3a8126f9"
server_key_iv: "2b2a12ff4262a5904e8657d3971d24ec"
server_key_version: "encryptionkey@0"
I20260807 06:33:38.203512 11728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:38.219290 11728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:38.219628 11728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:38.219785 11728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:38.223683 11728 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39165
I20260807 06:33:38.223752 11831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39165 every 8 connection(s)
I20260807 06:33:38.224084 11728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:38.225425 11832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:38.227708 11832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507: Bootstrap starting.
I20260807 06:33:38.228252 11832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:38.228482 11832 log.cc:824] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:38.229285 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 11728
I20260807 06:33:38.229333 11832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507: No bootstrap required, opened a new log
I20260807 06:33:38.229406 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:38.229604 3020 external_mini_cluster.cc:1428] Setting key 64815685f2d7b698c118739010ab0cd3
I20260807 06:33:38.232236 11832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39165 } }
I20260807 06:33:38.232434 11832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:38.232471 11832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6383e61222ae4a35bcd3f8fa6ca4e507, State: Initialized, Role: FOLLOWER
I20260807 06:33:38.232637 11832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39165 } }
I20260807 06:33:38.232724 11832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:38.232764 11832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:38.232813 11832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:38.233536 11832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39165 } }
I20260807 06:33:38.233632 11832 leader_election.cc:302] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6383e61222ae4a35bcd3f8fa6ca4e507; no voters:
I20260807 06:33:38.233871 11832 leader_election.cc:288] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:38.233902 11841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:38.234093 11841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [term 1 LEADER]: Becoming Leader. State: Replica: 6383e61222ae4a35bcd3f8fa6ca4e507, State: Running, Role: LEADER
I20260807 06:33:38.234162 11832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:38.234186 11841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39165 } }
I20260807 06:33:38.234508 11841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39165 } } }
I20260807 06:33:38.234570 11841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:38.234723 11841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6383e61222ae4a35bcd3f8fa6ca4e507. Latest consensus state: current_term: 1 leader_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6383e61222ae4a35bcd3f8fa6ca4e507" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39165 } } }
I20260807 06:33:38.234800 11841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:38.234902 11848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:38.235564 11848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:38.235579 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:38.230122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.2.243.62:39165)
0807 06:33:38.230323 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:38.230330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:38.230355 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:33:38.230823 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:38.230829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:38.230882 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:38.230991 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:38.231263 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.232124 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.233343 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.233858 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.233997 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:38.234740 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:38.234745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:38.234745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:33:38.234951 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:38.234995 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:33:38.235010 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:38.235082 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:38.235287 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 06:33:38.235373 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260807 06:33:38.236783 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:33:38.237752 11848 catalog_manager.cc:1347] Generated new cluster ID: 5eaf5700ebb64e30a67c6c4777145ea1
I20260807 06:33:38.237803 11848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:38.257433 11848 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:38.258204 11848 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:38.261240 11848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6383e61222ae4a35bcd3f8fa6ca4e507: Generated new TSK 0
I20260807 06:33:38.261466 11848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:33:38.291188 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:38.272185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35162 (local address 127.2.243.62:39165)
0807 06:33:38.272247 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:38.272249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:38.272367 (+ 118us) server_negotiation.cc:407] Connection header received
0807 06:33:38.272440 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:38.272441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:38.272453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:38.272557 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:33:38.272691 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.273167 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.274295 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.274498 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.274553 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:38.274730 (+ 177us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:33:38.290962 (+ 16232us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 06:33:38.291123 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35162: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 06:33:38.291271 11838 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35162: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 06:33:38.291435 11884 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:38.272119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39165 (local address 127.0.0.1:35162)
0807 06:33:38.272296 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:38.272307 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:38.272372 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:38.272568 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:38.272569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:38.272571 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:33:38.272634 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:38.272637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.273246 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.273247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:38.274209 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:38.274212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:38.274575 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:38.274576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:38.274635 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:38.274640 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:33:38.291118 (+ 16478us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 06:33:38.291251 (+ 133us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.243.62:39165: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":115,"threads_started":1}
I20260807 06:33:38.291816 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 11728
2026-08-07T06:33:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (225 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:38 dist-test-slave-rwrg krb5kdc[11903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:38 dist-test-slave-rwrg krb5kdc[11903](info): set up 2 sockets
Aug 07 06:33:38 dist-test-slave-rwrg krb5kdc[11903](info): commencing operation
krb5kdc: starting...
W20260807 06:33:39.742408 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:39 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:39.868563 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42675
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:39.952921 11921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:39.953156 11921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:39.953194 11921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:39.954361 11921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:39.954428 11921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:39.954453 11921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:39.954485 11921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:39.954519 11921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:39.955979 11921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:39.956259 11921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:39.956542 11921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:39.961483 11932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:39.961481 11936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:39.961589 11933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:39.961956 11921 server_base.cc:1034] running on GCE node
I20260807 06:33:39.962270 11921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:39.962553 11921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:39.963701 11921 hybrid_clock.cc:648] HybridClock initialized: now 1786084419963686 us; error 30 us; skew 500 ppm
Aug 07 06:33:39 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084419, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:39.966233 11921 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:39.966698 11921 webserver.cc:460] Webserver started at http://127.2.243.62:38943/ using document root <none> and password file <none>
I20260807 06:33:39.966922 11921 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:39.966990 11921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:39.967147 11921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:39.968192 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "010cd5d005994b648c903d9d4f0f0a91"
format_stamp: "Formatted at 2026-08-07 06:33:39 on dist-test-slave-rwrg"
server_key: "97f694f92d23049305172a5005cb37d3"
server_key_iv: "0118ca712e5532d223b86832e4dc32ed"
server_key_version: "encryptionkey@0"
I20260807 06:33:39.968628 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "010cd5d005994b648c903d9d4f0f0a91"
format_stamp: "Formatted at 2026-08-07 06:33:39 on dist-test-slave-rwrg"
server_key: "97f694f92d23049305172a5005cb37d3"
server_key_iv: "0118ca712e5532d223b86832e4dc32ed"
server_key_version: "encryptionkey@0"
I20260807 06:33:39.969959 11921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:39.970677 11954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:39.970840 11921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:39.970940 11921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "010cd5d005994b648c903d9d4f0f0a91"
format_stamp: "Formatted at 2026-08-07 06:33:39 on dist-test-slave-rwrg"
server_key: "97f694f92d23049305172a5005cb37d3"
server_key_iv: "0118ca712e5532d223b86832e4dc32ed"
server_key_version: "encryptionkey@0"
I20260807 06:33:39.971037 11921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:39.989710 11921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:39.991667 11921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:39.991853 11921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:39.996163 11921 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42675
I20260807 06:33:39.996208 12035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42675 every 8 connection(s)
I20260807 06:33:39.996661 11921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:39.997623 12036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:39.999948 12036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91: Bootstrap starting.
I20260807 06:33:40.000618 12036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:40.000912 12036 log.cc:824] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:40.001642 12036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91: No bootstrap required, opened a new log
I20260807 06:33:40.003257 12036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd5d005994b648c903d9d4f0f0a91" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42675 } }
I20260807 06:33:40.003394 12036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:40.003417 12036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010cd5d005994b648c903d9d4f0f0a91, State: Initialized, Role: FOLLOWER
I20260807 06:33:40.003593 12036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd5d005994b648c903d9d4f0f0a91" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42675 } }
I20260807 06:33:40.003688 12036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:40.003723 12036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:40.003782 12036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:40.003839 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 11921
I20260807 06:33:40.003943 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:40.004149 3020 external_mini_cluster.cc:1428] Setting key bddcbed307092eb92f3d007a2fe11df9
I20260807 06:33:40.004556 12036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd5d005994b648c903d9d4f0f0a91" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42675 } }
I20260807 06:33:40.004668 12036 leader_election.cc:302] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 010cd5d005994b648c903d9d4f0f0a91; no voters:
I20260807 06:33:40.004844 12036 leader_election.cc:288] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:40.005105 12036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:40.005239 12046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:40.005436 12046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [term 1 LEADER]: Becoming Leader. State: Replica: 010cd5d005994b648c903d9d4f0f0a91, State: Running, Role: LEADER
I20260807 06:33:40.005573 12046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd5d005994b648c903d9d4f0f0a91" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42675 } }
I20260807 06:33:40.006286 12048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 010cd5d005994b648c903d9d4f0f0a91. Latest consensus state: current_term: 1 leader_uuid: "010cd5d005994b648c903d9d4f0f0a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd5d005994b648c903d9d4f0f0a91" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42675 } } }
I20260807 06:33:40.006386 12048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:40.006768 12064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:40.007048 12045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "010cd5d005994b648c903d9d4f0f0a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010cd5d005994b648c903d9d4f0f0a91" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42675 } } }
I20260807 06:33:40.007125 12045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:40.007409 12064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:40.008988 12064 catalog_manager.cc:1347] Generated new cluster ID: e6cc8a8f303841029dbbf8539432eb2a
I20260807 06:33:40.009060 12064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:40.016788 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.004686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.2.243.62:42675)
0807 06:33:40.004875 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.004886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.004923 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:33:40.005068 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.005078 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.005139 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.005296 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.006271 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.007755 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.008816 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.009416 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.009518 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.012369 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:40.012373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:40.012373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:40.012385 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:40.015099 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.015494 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.015498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.015498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.015623 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.015829 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.015830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.015831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.016159 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:40.016314 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:40.016388 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 06:33:40.016576 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260807 06:33:40.022190 12064 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:40.022691 12064 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:40.028102 12064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 010cd5d005994b648c903d9d4f0f0a91: Generated new TSK 0
I20260807 06:33:40.028306 12064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:40.059332 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42675
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:40.143481 12079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:40.143673 12079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:40.143695 12079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:40.144927 12079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:40.144971 12079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:40.145006 12079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:40.146469 12079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:40.146708 12079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:40.146938 12079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:40.152194 12088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:40.152196 12094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:40.152221 12090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:40.152696 12079 server_base.cc:1034] running on GCE node
I20260807 06:33:40.152956 12079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:40.153199 12079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:40.154420 12079 hybrid_clock.cc:648] HybridClock initialized: now 1786084420154385 us; error 26 us; skew 500 ppm
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.157681 12079 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:40.158219 12079 webserver.cc:460] Webserver started at http://127.2.243.1:39905/ using document root <none> and password file <none>
I20260807 06:33:40.158506 12079 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:40.158562 12079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:40.158725 12079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:40.159850 12079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "389654f8c29349d6a336f47cf07652ba"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "e715fa970b561b0622e9f5f06096b43d"
server_key_iv: "c27e9d1b718d6fc43c5d86558bba464b"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.160324 12079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "389654f8c29349d6a336f47cf07652ba"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "e715fa970b561b0622e9f5f06096b43d"
server_key_iv: "c27e9d1b718d6fc43c5d86558bba464b"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.161913 12079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:40.162760 12106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.162941 12079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:40.163033 12079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "389654f8c29349d6a336f47cf07652ba"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "e715fa970b561b0622e9f5f06096b43d"
server_key_iv: "c27e9d1b718d6fc43c5d86558bba464b"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.163084 12079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:40.188428 12079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:40.190575 12079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:40.190688 12079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:40.190877 12079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:40.191210 12079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:40.191246 12079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.191268 12079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:40.191282 12079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.197780 12079 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:36161
I20260807 06:33:40.197860 12284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:36161 every 8 connection(s)
I20260807 06:33:40.198191 12079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:40.204900 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 12079
I20260807 06:33:40.204990 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:40.205217 3020 external_mini_cluster.cc:1428] Setting key cd3fd0bd217c312c08c3dfda4abc9e17
I20260807 06:33:40.208012 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.198984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:57095 (local address 127.2.243.62:42675)
0807 06:33:40.199058 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.199061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.199224 (+ 163us) server_negotiation.cc:407] Connection header received
0807 06:33:40.200141 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.200142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.200154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.200217 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.200945 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.201443 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.202427 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.202657 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.202716 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.204794 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:40.204796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:40.204796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:40.204804 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:40.206521 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.206973 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.206975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.206975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.207050 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.207319 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.207320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.207320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.207438 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:40.207516 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:40.207877 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 06:33:40.207947 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:33:40.208303 12288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.198929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42675 (local address 127.2.243.1:57095)
0807 06:33:40.199095 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:40.199137 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:40.200017 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:40.200291 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:40.200294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:40.200661 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:40.200848 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:40.200853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.201558 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.201558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:40.202344 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:40.202345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.202736 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.202737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:40.202834 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.203274 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:40.203282 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:40.204698 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:40.206599 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:40.206606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:40.206609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:40.206890 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:40.207119 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:40.207120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:40.207121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:40.207242 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:40.207514 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:40.207522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:40.207714 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 06:33:40.207837 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 06:33:40.208023 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260807 06:33:40.208949 12286 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42675
I20260807 06:33:40.209053 12286 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:40.209268 12286 heartbeater.cc:511] Master 127.2.243.62:42675 requested a full tablet report, sending...
I20260807 06:33:40.209801 11980 ts_manager.cc:194] Registered new tserver with Master: 389654f8c29349d6a336f47cf07652ba (127.2.243.1:36161)
I20260807 06:33:40.210435 11980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:57095
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:40.233882 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42675
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:40.315227 12291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:40.315418 12291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:40.315439 12291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:40.316550 12291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:40.316588 12291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:40.316612 12291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:40.318063 12291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:40.318338 12291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:40.318606 12291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:40.323618 12301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:40.323611 12298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:40.323640 12297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:40.323832 12291 server_base.cc:1034] running on GCE node
I20260807 06:33:40.324281 12291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:40.324599 12291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:40.325748 12291 hybrid_clock.cc:648] HybridClock initialized: now 1786084420325726 us; error 44 us; skew 500 ppm
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.328337 12291 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:40.328823 12291 webserver.cc:460] Webserver started at http://127.2.243.2:33951/ using document root <none> and password file <none>
I20260807 06:33:40.329061 12291 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:40.329128 12291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:40.329311 12291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:40.330343 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "27ca8904382e4f818a823cdc057ae238"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "e978939556bf0763036cc97c7f4aea65"
server_key_iv: "cfbf68c78c55ca2c3a6447ecd475986a"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.330721 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "27ca8904382e4f818a823cdc057ae238"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "e978939556bf0763036cc97c7f4aea65"
server_key_iv: "cfbf68c78c55ca2c3a6447ecd475986a"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.332053 12291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:40.332882 12310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.333057 12291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:33:40.333165 12291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "27ca8904382e4f818a823cdc057ae238"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "e978939556bf0763036cc97c7f4aea65"
server_key_iv: "cfbf68c78c55ca2c3a6447ecd475986a"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.333238 12291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:40.343824 12291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:40.346711 12291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:40.346868 12291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:40.347110 12291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:40.347479 12291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:40.347530 12291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.347563 12291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:40.347584 12291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.353669 12291 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:36173
I20260807 06:33:40.353742 12491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:36173 every 8 connection(s)
I20260807 06:33:40.354115 12291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:40.358955 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 12291
I20260807 06:33:40.359078 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:40.359263 3020 external_mini_cluster.cc:1428] Setting key c352b9bf7c952d492946e3565560c04f
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:40.364255 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.354862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:34089 (local address 127.2.243.62:42675)
0807 06:33:40.354948 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.354952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.355138 (+ 186us) server_negotiation.cc:407] Connection header received
0807 06:33:40.356099 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.356101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.356114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.356180 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.356865 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.357358 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.358464 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.358669 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.358753 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.361067 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:40.361068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:40.361069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:40.361076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:40.362770 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.363319 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.363319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.363320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.363371 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.363650 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.363651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.363651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.363751 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:40.363802 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:40.364130 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 06:33:40.364193 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:33:40.364617 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.354840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42675 (local address 127.2.243.2:34089)
0807 06:33:40.355018 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:40.355053 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:40.355971 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:40.356253 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:40.356256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:40.356572 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:40.356765 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:40.356770 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.357453 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.357455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:40.358349 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:40.358351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.358764 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.358783 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:40.358893 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.359422 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:40.359430 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:40.360946 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:40.362859 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:40.362865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:40.362866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:40.363234 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:40.363436 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:40.363437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:40.363437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:40.363581 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:40.363817 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:40.363823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:40.363986 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 06:33:40.364110 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:33:40.364297 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260807 06:33:40.365175 12493 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42675
I20260807 06:33:40.365290 12493 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:40.365466 12493 heartbeater.cc:511] Master 127.2.243.62:42675 requested a full tablet report, sending...
I20260807 06:33:40.365952 11980 ts_manager.cc:194] Registered new tserver with Master: 27ca8904382e4f818a823cdc057ae238 (127.2.243.2:36173)
I20260807 06:33:40.366503 11980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:34089
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:40.387544 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42675
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:40.471683 12501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:40.471952 12501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:40.471976 12501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:40.473242 12501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:40.473285 12501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:40.473333 12501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:40.474973 12501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:40.475238 12501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:40.475461 12501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:40.480410 12510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:40.480417 12514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:40.480439 12509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:40.480631 12501 server_base.cc:1034] running on GCE node
I20260807 06:33:40.480894 12501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:40.481166 12501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:40.482326 12501 hybrid_clock.cc:648] HybridClock initialized: now 1786084420482319 us; error 30 us; skew 500 ppm
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.484867 12501 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:40.485292 12501 webserver.cc:460] Webserver started at http://127.2.243.3:44297/ using document root <none> and password file <none>
I20260807 06:33:40.485481 12501 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:40.485512 12501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:40.485612 12501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:40.486605 12501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "f0fd5b3f22cd4a9087dbfb4408ff3845"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "5eaa537a5f02604e7825b265eaf219c6"
server_key_iv: "8f44023a0a1d49c1ec98dc137bda7fce"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.486897 12501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "f0fd5b3f22cd4a9087dbfb4408ff3845"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "5eaa537a5f02604e7825b265eaf219c6"
server_key_iv: "8f44023a0a1d49c1ec98dc137bda7fce"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.488095 12501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:33:40.488767 12528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.488969 12501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:33:40.489078 12501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "f0fd5b3f22cd4a9087dbfb4408ff3845"
format_stamp: "Formatted at 2026-08-07 06:33:40 on dist-test-slave-rwrg"
server_key: "5eaa537a5f02604e7825b265eaf219c6"
server_key_iv: "8f44023a0a1d49c1ec98dc137bda7fce"
server_key_version: "encryptionkey@0"
I20260807 06:33:40.489151 12501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:40.499536 12501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:40.501336 12501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:40.501493 12501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:40.501709 12501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:40.502027 12501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:40.502086 12501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.502127 12501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:40.502156 12501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:40.508792 12501 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:34945
I20260807 06:33:40.508843 12707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:34945 every 8 connection(s)
I20260807 06:33:40.509147 12501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:40.512930 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 12501
I20260807 06:33:40.513019 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:40.513257 3020 external_mini_cluster.cc:1428] Setting key 7480795075284a64520f984fc0d833ec
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:40.518563 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.509866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:39075 (local address 127.2.243.62:42675)
0807 06:33:40.509909 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.509912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.510140 (+ 228us) server_negotiation.cc:407] Connection header received
0807 06:33:40.511016 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.511017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.511027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.511099 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.511654 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.512138 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.513241 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.513525 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.513613 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.515883 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:40.515884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:40.515885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:40.515891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:40.517307 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.517682 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.517682 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.517683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.517741 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.517982 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.517984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.517984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.518106 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:40.518171 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:40.518398 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 06:33:40.518464 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:40.518694 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.509846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42675 (local address 127.2.243.3:39075)
0807 06:33:40.510003 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:40.510047 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:40.510929 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:40.511152 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:40.511156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:40.511401 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:40.511580 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:40.511584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.512214 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.512214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:40.513155 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:40.513158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.513662 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.513663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:40.513827 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.514394 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:40.514402 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:40.515798 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:40.517379 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:40.517385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:40.517387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:40.517605 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:40.517803 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:40.517804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:40.517804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:40.517898 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:40.518173 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:40.518177 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:40.518284 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 06:33:40.518365 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:33:40.518479 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":98,"threads_started":1}
I20260807 06:33:40.519230 12708 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42675
I20260807 06:33:40.519332 12708 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:40.519513 12708 heartbeater.cc:511] Master 127.2.243.62:42675 requested a full tablet report, sending...
I20260807 06:33:40.519976 11980 ts_manager.cc:194] Registered new tserver with Master: f0fd5b3f22cd4a9087dbfb4408ff3845 (127.2.243.3:34945)
I20260807 06:33:40.520323 11980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:39075
I20260807 06:33:40.525102 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/test-admin.keytab.
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.541829 3020 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 06:33:40.543479 12718 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:40.542666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.2.243.1:36161)
0807 06:33:40.542824 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.542829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.542839 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:40.542863 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.542865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.542878 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.542949 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.543339 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60850: BlockingRecv error: recv got EOF from 127.0.0.1:60850 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
W20260807 06:33:40.543584 3020 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5cc' of type FILE
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.544616 3020 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.545497 3020 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:40.553268 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.546010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.2.243.1:36161)
0807 06:33:40.546077 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.546079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.546105 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:33:40.546185 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.546185 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.546203 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.546262 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.546668 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.547404 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.548123 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.548432 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.548535 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.550862 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:40.550869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:40.550870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:40.550876 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:40.552230 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.552597 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.552599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.552600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.552667 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.552845 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.552846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.552846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.552970 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:40.553085 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:40.553138 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 06:33:40.553198 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 06:33:40.555045 12718 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:40.554482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.2.243.1:36161)
0807 06:33:40.554515 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.554517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.554555 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:33:40.554602 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.554602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.554610 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.554654 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.555014 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60878: BlockingRecv error: recv got EOF from 127.0.0.1:60878 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 06:33:40.555123 3020 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5cc' of type FILE
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.556053 3020 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.556968 3020 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:40.564003 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.557469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.2.243.1:36161)
0807 06:33:40.557529 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.557531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.557591 (+ 60us) server_negotiation.cc:407] Connection header received
0807 06:33:40.557674 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.557675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.557685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.557741 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.558092 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.558597 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.559283 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.559490 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.559571 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.561448 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:40.561449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:40.561450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:40.561457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:40.562776 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.563194 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.563196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.563196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.563261 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.563502 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.563503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.563503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.563605 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:40.563708 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:40.563831 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 06:33:40.563908 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 06:33:40.565557 12718 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:40.565005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60906 (local address 127.2.243.1:36161)
0807 06:33:40.565040 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.565042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.565091 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:33:40.565153 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.565154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.565160 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.565203 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.565509 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60906: BlockingRecv error: recv got EOF from 127.0.0.1:60906 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:33:40.565652 3020 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786084392481039-3020-0/krb5kdc/krb5cc' of type FILE
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.566629 3020 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:40.567495 3020 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[11903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:33:40.574230 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:40.568114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.2.243.1:36161)
0807 06:33:40.568175 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:40.568177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:40.568259 (+ 82us) server_negotiation.cc:407] Connection header received
0807 06:33:40.568330 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:40.568331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:40.568339 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:40.568409 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:40.568859 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.569323 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.569992 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:40.570184 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:40.570260 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:40.572190 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:40.572191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:40.572192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:40.572198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:40.573406 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.573697 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.573698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.573698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.573747 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:40.573895 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:40.573895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:40.573895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:40.573970 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:40.574032 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:40.574128 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 06:33:40.574178 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:33:40.574935 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 12079
I20260807 06:33:40.586702 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 12291
I20260807 06:33:40.604676 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 12501
I20260807 06:33:40.613865 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 11921
2026-08-07T06:33:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2129 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[12737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[12737](info): set up 2 sockets
Aug 07 06:33:40 dist-test-slave-rwrg krb5kdc[12737](info): commencing operation
krb5kdc: starting...
W20260807 06:33:41.895015 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:41 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab.
Entry for principal oryx/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab.
I20260807 06:33:42.020932 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42593
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab
--principal=oryx/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:42.102303 12760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:42.102540 12760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:42.102586 12760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:42.103715 12760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:42.103786 12760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:42.103811 12760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:42.103842 12760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:42.103873 12760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:42.105386 12760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab
--principal=oryx/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:42.105675 12760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:42.105919 12760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:42.110949 12775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.111013 12770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.111161 12760 server_base.cc:1034] running on GCE node
W20260807 06:33:42.111227 12769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.111495 12760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:42.111759 12760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:42.112903 12760 hybrid_clock.cc:648] HybridClock initialized: now 1786084422112893 us; error 35 us; skew 500 ppm
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:42.115923 12760 init.cc:377] Logged in from keytab as oryx/127.2.243.62@KRBTEST.COM (short username oryx)
I20260807 06:33:42.116366 12760 webserver.cc:460] Webserver started at http://127.2.243.62:40623/ using document root <none> and password file <none>
I20260807 06:33:42.116671 12760 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:42.116719 12760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:42.116881 12760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:42.117962 12760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "b9e3c8301267400e901954e239974667"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "c0c246c43abc8078cfc45b42776df72f"
server_key_iv: "19856089dae236020509857468c5870e"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.118347 12760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "b9e3c8301267400e901954e239974667"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "c0c246c43abc8078cfc45b42776df72f"
server_key_iv: "19856089dae236020509857468c5870e"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.119681 12760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:42.120440 12787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.120678 12760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:42.120788 12760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "b9e3c8301267400e901954e239974667"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "c0c246c43abc8078cfc45b42776df72f"
server_key_iv: "19856089dae236020509857468c5870e"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.120877 12760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:42.131858 12760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:42.134277 12760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:42.134444 12760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:42.138854 12760 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42593
I20260807 06:33:42.138888 12878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42593 every 8 connection(s)
I20260807 06:33:42.139259 12760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:42.140285 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:42.142884 12879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: Bootstrap starting.
I20260807 06:33:42.143481 12879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:42.143776 12879 log.cc:824] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:42.144534 12879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: No bootstrap required, opened a new log
I20260807 06:33:42.145957 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 12760
I20260807 06:33:42.146080 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:42.146301 3020 external_mini_cluster.cc:1428] Setting key eae86cee1096aa52e5ee71685d47dd05
I20260807 06:33:42.146241 12879 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } }
I20260807 06:33:42.146406 12879 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:42.146461 12879 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e3c8301267400e901954e239974667, State: Initialized, Role: FOLLOWER
I20260807 06:33:42.146653 12879 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } }
I20260807 06:33:42.146755 12879 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:42.146788 12879 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:42.146842 12879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:42.147467 12879 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } }
I20260807 06:33:42.147591 12879 leader_election.cc:302] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9e3c8301267400e901954e239974667; no voters:
I20260807 06:33:42.147755 12879 leader_election.cc:288] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:42.147794 12888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:42.147910 12888 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 1 LEADER]: Becoming Leader. State: Replica: b9e3c8301267400e901954e239974667, State: Running, Role: LEADER
I20260807 06:33:42.148033 12879 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:42.148063 12888 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } }
I20260807 06:33:42.149430 12889 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } }
I20260807 06:33:42.149547 12889 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:42.149978 12891 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e3c8301267400e901954e239974667. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } }
I20260807 06:33:42.150100 12891 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
W20260807 06:33:42.150285 12910 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:33:42.150345 12910 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:33:42.150391 12912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:42.150533 12912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:42.152472 12912 catalog_manager.cc:1347] Generated new cluster ID: 42e49fac30c54cb99f4487985a373342
I20260807 06:33:42.152549 12912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:42.157785 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.146823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58816 (local address 127.2.243.62:42593)
0807 06:33:42.147023 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.147033 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.147073 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:33:42.147417 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.147423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.147466 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.147633 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.148434 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.149993 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.151050 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.151661 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.151784 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.154386 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.154390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.154390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.154403 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.156155 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.156586 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.156589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.156589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.156706 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.156926 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.156927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.156927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.157177 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.157363 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.157431 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.157577 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260807 06:33:42.181618 12912 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:42.182175 12912 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:42.191318 12912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: Generated new TSK 0
I20260807 06:33:42.191524 12912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab.
Entry for principal oryx/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab.
I20260807 06:33:42.225594 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42593
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab
--principal=oryx/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:42.311658 12918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:42.311858 12918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:42.311882 12918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:42.313110 12918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:42.313154 12918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:42.313180 12918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:42.314664 12918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab
--principal=oryx/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:42.314925 12918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:42.315168 12918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:42.320058 12932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.320223 12918 server_base.cc:1034] running on GCE node
W20260807 06:33:42.320070 12925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.320067 12924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.320572 12918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:42.320851 12918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:42.322008 12918 hybrid_clock.cc:648] HybridClock initialized: now 1786084422321990 us; error 33 us; skew 500 ppm
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:42.327656 12918 init.cc:377] Logged in from keytab as oryx/127.2.243.1@KRBTEST.COM (short username oryx)
I20260807 06:33:42.328130 12918 webserver.cc:460] Webserver started at http://127.2.243.1:40419/ using document root <none> and password file <none>
I20260807 06:33:42.328380 12918 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:42.328449 12918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:42.328620 12918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:42.329707 12918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "9be736c4bcf346bbb551433af3b5cb57"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "31ffa6178d11825acc444fc56a4c5c1d"
server_key_iv: "2bfe202b3a9d2acd3aac9f76b52a8bb7"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.330072 12918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "9be736c4bcf346bbb551433af3b5cb57"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "31ffa6178d11825acc444fc56a4c5c1d"
server_key_iv: "2bfe202b3a9d2acd3aac9f76b52a8bb7"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.331470 12918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 06:33:42.332254 12943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.332473 12918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:42.332619 12918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "9be736c4bcf346bbb551433af3b5cb57"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "31ffa6178d11825acc444fc56a4c5c1d"
server_key_iv: "2bfe202b3a9d2acd3aac9f76b52a8bb7"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.332697 12918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:42.344326 12918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:42.346737 12918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:42.346908 12918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:42.347146 12918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:42.347466 12918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:42.347527 12918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.347571 12918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:42.347597 12918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.354681 12918 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40539
I20260807 06:33:42.354717 13130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40539 every 8 connection(s)
I20260807 06:33:42.355155 12918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:42.361217 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 12918
I20260807 06:33:42.361302 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:42.361500 3020 external_mini_cluster.cc:1428] Setting key 1bd58c3da73ba870e66e65ef40667637
I20260807 06:33:42.365159 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.356000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:56795 (local address 127.2.243.62:42593)
0807 06:33:42.356068 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.356071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.356314 (+ 243us) server_negotiation.cc:407] Connection header received
0807 06:33:42.356859 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.356860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.356870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.356924 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.357825 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.358326 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.359286 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.359470 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.359555 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.361880 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.361882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.361883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.361890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.363640 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.364060 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.364061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.364062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.364129 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.364395 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.364396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.364396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.364589 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.364678 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.364957 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.365049 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:33:42.365346 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.355981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:56795)
0807 06:33:42.356181 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.356218 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.356767 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.356974 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.356978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.357544 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.357748 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.357752 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.358405 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.358406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.359215 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.359217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.359569 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.359591 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.359690 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.360176 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.360184 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.361792 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.363725 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.363732 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.363734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.363958 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.364200 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.364201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.364201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.364319 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.364668 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.364675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.364821 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.364933 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.365095 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":114,"threads_started":1}
I20260807 06:33:42.365835 13131 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42593
I20260807 06:33:42.365947 13131 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:42.366125 13131 heartbeater.cc:511] Master 127.2.243.62:42593 requested a full tablet report, sending...
I20260807 06:33:42.366748 12822 ts_manager.cc:194] Registered new tserver with Master: 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1:40539)
I20260807 06:33:42.367722 12822 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:56795
WARNING: no policy specified for oryx/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab.
Entry for principal oryx/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab.
I20260807 06:33:42.389770 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42593
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab
--principal=oryx/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:42.472860 13139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:42.473065 13139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:42.473090 13139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:42.474181 13139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:42.474213 13139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:42.474238 13139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:42.475605 13139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab
--principal=oryx/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:42.475782 13139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:42.475960 13139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:42.480965 13149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.481062 13139 server_base.cc:1034] running on GCE node
W20260807 06:33:42.480978 13152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.480990 13148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.481415 13139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:42.481616 13139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:42.482752 13139 hybrid_clock.cc:648] HybridClock initialized: now 1786084422482730 us; error 38 us; skew 500 ppm
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:42.485273 13139 init.cc:377] Logged in from keytab as oryx/127.2.243.2@KRBTEST.COM (short username oryx)
I20260807 06:33:42.485733 13139 webserver.cc:460] Webserver started at http://127.2.243.2:37437/ using document root <none> and password file <none>
I20260807 06:33:42.485957 13139 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:42.486004 13139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:42.486153 13139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:42.487227 13139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "7bfcc9cc7f452c3b8418cd1f9bbb1fce"
server_key_iv: "bece444551c58923f299befb83e8db06"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.487588 13139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "7bfcc9cc7f452c3b8418cd1f9bbb1fce"
server_key_iv: "bece444551c58923f299befb83e8db06"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.488948 13139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:42.489780 13165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.490020 13139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:42.490140 13139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "7bfcc9cc7f452c3b8418cd1f9bbb1fce"
server_key_iv: "bece444551c58923f299befb83e8db06"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.490231 13139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:42.500730 13139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:42.502658 13139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:42.502833 13139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:42.503077 13139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:42.503396 13139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:42.503455 13139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.503531 13139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:42.503566 13139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.510241 13139 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:38873
I20260807 06:33:42.510277 13340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:38873 every 8 connection(s)
I20260807 06:33:42.510636 13139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:42.515288 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 13139
I20260807 06:33:42.515378 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:42.515573 3020 external_mini_cluster.cc:1428] Setting key 51d6e3e6556f0611ae32e735b19135e4
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:42.520707 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.511451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:47259 (local address 127.2.243.62:42593)
0807 06:33:42.511521 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.511523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.511775 (+ 252us) server_negotiation.cc:407] Connection header received
0807 06:33:42.512246 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.512247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.512257 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.512321 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.513300 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.513806 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.514861 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.515106 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.515189 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.517714 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.517716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.517716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.517724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.519180 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.519686 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.519687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.519688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.519745 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.520026 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.520026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.520027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.520129 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.520182 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.520514 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.520621 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:33:42.520995 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.511427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:47259)
0807 06:33:42.511612 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.511655 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.512159 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.512374 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.512377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.513004 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.513199 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.513204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.513910 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.513911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.514762 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.514765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.515202 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.515203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.515306 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.515979 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.515988 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.517603 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.519279 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.519286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.519289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.519594 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.519814 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.519815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.519815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.519958 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.520195 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.520200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.520361 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.520470 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.520708 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260807 06:33:42.521596 13341 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42593
I20260807 06:33:42.521690 13341 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:42.521885 13341 heartbeater.cc:511] Master 127.2.243.62:42593 requested a full tablet report, sending...
I20260807 06:33:42.522274 12822 ts_manager.cc:194] Registered new tserver with Master: 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2:38873)
I20260807 06:33:42.522657 12822 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:47259
WARNING: no policy specified for oryx/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab.
Entry for principal oryx/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab.
I20260807 06:33:42.542815 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42593
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab
--principal=oryx/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:42.627409 13348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:42.627625 13348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:42.627650 13348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:42.628867 13348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:42.628908 13348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:42.628934 13348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:42.630411 13348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab
--principal=oryx/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:42.630662 13348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:42.630884 13348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:42.636030 13366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.635968 13360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.635977 13357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.635994 13348 server_base.cc:1034] running on GCE node
I20260807 06:33:42.636430 13348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:42.636781 13348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:42.637946 13348 hybrid_clock.cc:648] HybridClock initialized: now 1786084422637936 us; error 27 us; skew 500 ppm
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:42.640458 13348 init.cc:377] Logged in from keytab as oryx/127.2.243.3@KRBTEST.COM (short username oryx)
I20260807 06:33:42.641026 13348 webserver.cc:460] Webserver started at http://127.2.243.3:42531/ using document root <none> and password file <none>
I20260807 06:33:42.641302 13348 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:42.641374 13348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:42.641526 13348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:42.642606 13348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "48b2a453116349bca087486129d966f8"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "5c45667bac63bb21e0137f676bc14610"
server_key_iv: "e9cc247952c1e6e1264caad9419aa482"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.642998 13348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "48b2a453116349bca087486129d966f8"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "5c45667bac63bb21e0137f676bc14610"
server_key_iv: "e9cc247952c1e6e1264caad9419aa482"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.644436 13348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:42.645282 13374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.645473 13348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:42.645607 13348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "48b2a453116349bca087486129d966f8"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "5c45667bac63bb21e0137f676bc14610"
server_key_iv: "e9cc247952c1e6e1264caad9419aa482"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.645685 13348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:42.667016 13348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:42.669520 13348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:42.669691 13348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:42.669943 13348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:42.670270 13348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:42.670328 13348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.670373 13348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:42.670399 13348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.676326 13348 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:38203
I20260807 06:33:42.676368 13548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:38203 every 8 connection(s)
I20260807 06:33:42.676789 13348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:42.678241 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 13348
I20260807 06:33:42.678355 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:42.678618 3020 external_mini_cluster.cc:1428] Setting key 766f4c518649910bca39554d41eb6c3a
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:42.686565 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.677544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:38351 (local address 127.2.243.62:42593)
0807 06:33:42.677630 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.677633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.677873 (+ 240us) server_negotiation.cc:407] Connection header received
0807 06:33:42.678630 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.678631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.678642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.678730 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.680024 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.680531 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.681586 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.681765 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.681818 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.684063 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.684064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.684064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.684071 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.685331 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.685728 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.685729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.685729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.685778 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.686041 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.686041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.686041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.686159 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.686208 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.686447 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.686504 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:33:42.686805 13551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.677498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:38351)
0807 06:33:42.677733 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.677774 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.678511 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.678828 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.678832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.679705 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.679928 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.679933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.680649 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.680650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.681508 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.681510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.681851 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.681851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.681957 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.682549 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.682559 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.683948 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.685407 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.685413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.685415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.685657 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.685840 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.685840 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.685841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.685944 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.686217 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.686222 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.686334 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.686421 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.686542 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
I20260807 06:33:42.687367 13549 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42593
I20260807 06:33:42.687485 13549 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:42.687664 13549 heartbeater.cc:511] Master 127.2.243.62:42593 requested a full tablet report, sending...
I20260807 06:33:42.688105 12822 ts_manager.cc:194] Registered new tserver with Master: 48b2a453116349bca087486129d966f8 (127.2.243.3:38203)
I20260807 06:33:42.688457 12822 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:38351
I20260807 06:33:42.690402 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.243.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.61.keytab.
Entry for principal oryx/127.2.243.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.61.keytab.
I20260807 06:33:42.717222 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:37469
--webserver_interface=127.2.243.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.61.keytab
--principal=oryx/127.2.243.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42593,127.2.243.61:37469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:42.801199 13558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:42.801421 13558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:42.801445 13558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:42.802573 13558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:42.802609 13558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:42.802623 13558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:42.802636 13558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:42.802649 13558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:42.804046 13558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42593,127.2.243.61:37469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.61.keytab
--principal=oryx/127.2.243.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:37469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.243.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
W20260807 06:33:42.804148 13558 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.243.62:42593,127.2.243.61:37469'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 06:33:42.804260 13558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:42.804422 13558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:42.809401 13569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.809394 13570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.809418 13573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.809777 13558 server_base.cc:1034] running on GCE node
I20260807 06:33:42.810039 13558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:42.810352 13558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:42.811509 13558 hybrid_clock.cc:648] HybridClock initialized: now 1786084422811497 us; error 36 us; skew 500 ppm
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:42.814348 13558 init.cc:377] Logged in from keytab as oryx/127.2.243.61@KRBTEST.COM (short username oryx)
I20260807 06:33:42.814837 13558 webserver.cc:460] Webserver started at http://127.2.243.61:35701/ using document root <none> and password file <none>
I20260807 06:33:42.815065 13558 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:42.815133 13558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:42.815286 13558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:42.816339 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/instance:
uuid: "1bb181eac61946a58c33dcfdc076a5b2"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "23f7fdca35da3d4dd9addab71303e7d7"
server_key_iv: "b6827ee0a4cc04e97ebe001fd9d6fa2a"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.816737 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal/instance:
uuid: "1bb181eac61946a58c33dcfdc076a5b2"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "23f7fdca35da3d4dd9addab71303e7d7"
server_key_iv: "b6827ee0a4cc04e97ebe001fd9d6fa2a"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.818068 13558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:42.818871 13589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.819048 13558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:42.819159 13558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
uuid: "1bb181eac61946a58c33dcfdc076a5b2"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "23f7fdca35da3d4dd9addab71303e7d7"
server_key_iv: "b6827ee0a4cc04e97ebe001fd9d6fa2a"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.819231 13558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:42.833719 13558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:42.836079 13558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:42.836256 13558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:42.840356 13558 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.61:37469
I20260807 06:33:42.840410 13676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.61:37469 every 8 connection(s)
I20260807 06:33:42.840796 13558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
I20260807 06:33:42.841975 13677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:42.842703 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 13558
I20260807 06:33:42.844112 13677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.61@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.243.61@KRBTEST.COM
I20260807 06:33:42.854185 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.844918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.2.243.62:42593)
0807 06:33:42.844969 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.844971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.845150 (+ 179us) server_negotiation.cc:407] Connection header received
0807 06:33:42.845221 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.845222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.845236 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.845309 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.845787 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.846504 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.847798 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.848087 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.848149 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.850744 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.850745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.850745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.850752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.852588 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.853057 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.853059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.853059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.853122 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.853374 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.853375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.853375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.853543 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.853596 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.853994 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.854082 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 06:33:42.854423 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.844816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58826)
0807 06:33:42.845019 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.845075 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.845155 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.845325 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.845327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.845593 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.845693 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.845697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.846606 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.846607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.847715 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.847718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.848234 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.848236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.848340 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.848847 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.848875 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.850649 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.852680 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.852687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.852690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.852979 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.853200 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.853200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.853201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.853305 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.853622 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.853629 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.853809 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.853947 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.854130 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260807 06:33:42.855163 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.843445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53296 (local address 127.2.243.61:37469)
0807 06:33:42.843634 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.843642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.843670 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:33:42.844061 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.844066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.844109 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.844221 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.845139 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.846323 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.847488 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.847925 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.848027 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.851363 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.851371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.851371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.851379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.853321 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.853748 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.853750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.853751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.853801 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.854109 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.854110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.854110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.854278 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.854431 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.854939 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.855044 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":62,"threads_started":1}
I20260807 06:33:42.855278 13677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.61@KRBTEST.COM for oryx/127.2.243.61@KRBTEST.COM
I20260807 06:33:42.862353 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.855576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.0.0.1:53310)
0807 06:33:42.855622 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.855633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.855672 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.856092 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.856092 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.856202 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.856248 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.856250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.857009 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.857011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.857482 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.857483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.857863 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.857864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.857920 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.858319 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.858323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.859478 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.861282 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.861283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.861284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.861461 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.861692 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.861693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.861694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.861753 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.862019 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.862020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.862060 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.862136 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.862239 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 06:33:42.862416 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.855629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53310 (local address 127.2.243.61:37469)
0807 06:33:42.855966 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.855968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.855976 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:42.855995 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.855996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.856007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.856086 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.856317 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.856902 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.857572 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.857778 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.857837 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.859595 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.859596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.859597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.859604 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.861174 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.861560 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.861561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.861562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.861621 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.861833 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.861834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.861834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.861942 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.862036 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.862168 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.862292 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":316}
I20260807 06:33:42.863842 13677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Bootstrap starting.
I20260807 06:33:42.864768 13677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:42.865048 13677 log.cc:824] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:42.865862 13677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: No bootstrap required, opened a new log
I20260807 06:33:42.867223 13677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } }
I20260807 06:33:42.867363 13677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:42.867425 13677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bb181eac61946a58c33dcfdc076a5b2, State: Initialized, Role: FOLLOWER
I20260807 06:33:42.867584 13677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } }
I20260807 06:33:42.867882 13677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:42.867871 13686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } } }
I20260807 06:33:42.868000 13686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:33:42.869797 13704 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:33:42.869859 13704 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:33:42.871694 13558 master_runner.cc:428] Detected that this master 1bb181eac61946a58c33dcfdc076a5b2 is joining an existing cluster
I20260807 06:33:42.871763 13558 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.243.61:37469
I20260807 06:33:42.879730 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.873000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.2.243.62:42593)
0807 06:33:42.873069 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.873072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.873084 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:33:42.873116 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.873117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.873129 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.873215 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.873629 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.874228 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.875226 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.875525 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.875565 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.876802 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.876803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.876804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.876812 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.878378 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.878877 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.878878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.878879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.878937 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.879158 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.879159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.879159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.879293 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.879354 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.879551 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.879636 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:33:42.879737 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.872810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58828)
0807 06:33:42.872980 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.872992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.873060 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.873215 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.873215 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.873444 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.873534 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.873536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.874302 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.874303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.875131 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.875133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.875653 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.875653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.875702 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.876194 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.876199 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.876711 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.878485 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.878486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.878486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.878799 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.879025 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.879026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.879026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.879090 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.879379 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.879380 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.879422 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.879514 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.879618 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260807 06:33:42.879985 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.873387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.2.243.61:37469)
0807 06:33:42.873421 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.873423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.873430 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:33:42.873484 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.873484 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.873493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.873558 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.873900 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.874616 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.875583 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.875873 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.875949 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.876962 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.876963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.876964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.876970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.878834 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.879220 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.879221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.879222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.879264 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.879477 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.879478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.879478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.879593 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.879654 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.879806 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.879891 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:42.880157 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.872813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.0.0.1:53326)
0807 06:33:42.873373 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.873384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.873431 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.873561 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.873561 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.873749 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.873815 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.873817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.874717 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.874718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.875488 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.875489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.875957 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.875958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.876036 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.876403 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.876408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.876874 (+ 466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.878922 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.878923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.878924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.879135 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.879336 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.879337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.879338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.879405 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.879676 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.879677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.879708 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.880041 (+ 333us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.880099 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":487,"threads_started":1}
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.62@KRBTEST.COM for oryx/127.2.243.61@KRBTEST.COM
I20260807 06:33:42.890285 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.882855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53332 (local address 127.2.243.61:37469)
0807 06:33:42.882897 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.882899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.883048 (+ 149us) server_negotiation.cc:407] Connection header received
0807 06:33:42.883129 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.883130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.883138 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.883184 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.883670 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.884112 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.885229 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.885394 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.885445 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.887645 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.887647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.887647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.887654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.889049 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.889475 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.889477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.889477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.889537 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.889749 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.889750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.889750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.889853 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.889924 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.890118 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.890191 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:33:42.890321 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.882758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.0.0.1:53332)
0807 06:33:42.882935 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.882977 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.883056 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.883190 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.883193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.883538 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.883595 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.883598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.884192 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.884193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.885151 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.885154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.885490 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.885492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.885580 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.885983 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.885988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.887554 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.889165 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.889173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.889175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.889382 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.889613 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.889614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.889614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.889679 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.889933 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.889941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.889990 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.890091 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.890204 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260807 06:33:42.890872 12822 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.243.61:37469: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2"
member_type: NON_VOTER
last_known_addr {
host: "127.2.243.61"
port: 37469
}
attrs {
promote: true
}
}
dest_uuid: "b9e3c8301267400e901954e239974667"
cas_config_opid_index: -1
I20260807 06:33:42.891273 12822 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } }
I20260807 06:33:42.891812 13718 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } }
I20260807 06:33:42.892115 13640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:42.893023 13718 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.243.61:37469
I20260807 06:33:42.893044 13640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Refusing update from remote peer b9e3c8301267400e901954e239974667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 06:33:42.893332 13718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:42.893445 13558 master.cc:537] Master@127.2.243.61:37469 shutting down...
I20260807 06:33:42.893501 13717 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } }
I20260807 06:33:42.893967 13717 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
W20260807 06:33:42.894747 12788 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 06:33:42.895165 12788 consensus_peers.cc:487] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 -> Peer 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61:37469): Couldn't send request to peer 1bb181eac61946a58c33dcfdc076a5b2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 06:33:42.896621 13558 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:42.896711 13558 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:42.896781 13558 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: stopping tablet replica
I20260807 06:33:42.909502 13558 master.cc:552] Master@127.2.243.61:37469 shutdown complete.
I20260807 06:33:42.909554 13558 master_runner.cc:305] Clearing existing system tablet
I20260807 06:33:42.909703 13558 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:42.909754 13558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:42.910403 13558 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:42.911224 13723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.911357 13558 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:42.911418 13558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:33:42.911441 13558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
uuid: "1bb181eac61946a58c33dcfdc076a5b2"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "23f7fdca35da3d4dd9addab71303e7d7"
server_key_iv: "b6827ee0a4cc04e97ebe001fd9d6fa2a"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.912197 13558 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:42.913887 13558 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 06:33:42.913947 13558 log.cc:1196] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 06:33:42.914197 13558 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Deleting consensus metadata
I20260807 06:33:42.914332 13558 master_runner.cc:315] Copying system tablet from 127.2.243.62:42593
I20260807 06:33:42.915005 13558 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: tablet copy: Beginning tablet copy session from remote peer at address 127.2.243.62:42593
I20260807 06:33:42.922245 13732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.915392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58830)
0807 06:33:42.915609 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:42.915620 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:42.915691 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:42.915841 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:42.915842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:42.916033 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:42.916095 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.916096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.917076 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.917077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.917710 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:42.917714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.918195 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.918196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:42.918284 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.918832 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:42.918837 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:42.919157 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:42.921101 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.921102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.921103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.921285 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.921577 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:42.921578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:42.921578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:42.921667 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:42.921960 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:42.921961 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:42.922009 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:33:42.922067 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 06:33:42.922160 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
I20260807 06:33:42.922292 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.915523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.2.243.62:42593)
0807 06:33:42.915597 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.915600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.915693 (+ 93us) server_negotiation.cc:407] Connection header received
0807 06:33:42.915745 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.915746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.915757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.915826 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.916172 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.916976 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.917799 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.918111 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.918186 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.919253 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.919255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.919255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.919262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.921006 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.921373 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.921374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.921374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.921439 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.921743 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.921744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.921744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.921885 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.921958 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.922113 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.922197 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:33:42.922814 12854 tablet_copy_service.cc:140] P b9e3c8301267400e901954e239974667: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1bb181eac61946a58c33dcfdc076a5b2 ({username='oryx', principal='oryx/127.2.243.61@KRBTEST.COM'} at 127.0.0.1:58830)
I20260807 06:33:42.922961 12854 tablet_copy_service.cc:161] P b9e3c8301267400e901954e239974667: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1bb181eac61946a58c33dcfdc076a5b2 at {username='oryx', principal='oryx/127.2.243.61@KRBTEST.COM'} at 127.0.0.1:58830: session id = 1bb181eac61946a58c33dcfdc076a5b2-00000000000000000000000000000000
I20260807 06:33:42.923848 12854 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 06:33:42.924568 13558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 06:33:42.925704 13558 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: tablet copy: Starting download of 0 data blocks...
I20260807 06:33:42.925876 13558 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: tablet copy: Starting download of 1 WAL segments...
I20260807 06:33:42.927035 13558 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 06:33:42.927886 12854 tablet_copy_service.cc:342] P b9e3c8301267400e901954e239974667: Request end of tablet copy session 1bb181eac61946a58c33dcfdc076a5b2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.243.61@KRBTEST.COM'} at 127.0.0.1:58830
I20260807 06:33:42.927949 12854 tablet_copy_service.cc:433] P b9e3c8301267400e901954e239974667: ending tablet copy session 1bb181eac61946a58c33dcfdc076a5b2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1bb181eac61946a58c33dcfdc076a5b2
I20260807 06:33:42.930739 13558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:42.931879 13735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:42.931980 13737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.932228 13558 server_base.cc:1034] running on GCE node
W20260807 06:33:42.932303 13739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:42.932474 13558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:42.932652 13558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:42.933784 13558 hybrid_clock.cc:648] HybridClock initialized: now 1786084422933772 us; error 29 us; skew 500 ppm
Aug 07 06:33:42 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:42.935181 13558 init.cc:377] Logged in from keytab as oryx/127.2.243.61@KRBTEST.COM (short username oryx)
I20260807 06:33:42.935550 13558 webserver.cc:460] Webserver started at http://127.2.243.61:46831/ using document root <none> and password file <none>
I20260807 06:33:42.935695 13558 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:42.935743 13558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:42.936445 13558 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:42.937053 13749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.937180 13558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:42.937229 13558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
uuid: "1bb181eac61946a58c33dcfdc076a5b2"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "23f7fdca35da3d4dd9addab71303e7d7"
server_key_iv: "b6827ee0a4cc04e97ebe001fd9d6fa2a"
server_key_version: "encryptionkey@0"
I20260807 06:33:42.937289 13558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:42.944370 13558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:42.946038 13558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:42.949373 13558 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.61:37469
I20260807 06:33:42.949651 13836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.61:37469 every 8 connection(s)
I20260807 06:33:42.949846 13558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
I20260807 06:33:42.950804 13837 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:33:42.951354 13837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Bootstrap starting.
I20260807 06:33:42.954376 13837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:33:42.954634 13837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: Bootstrap complete.
I20260807 06:33:42.954852 13837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } }
I20260807 06:33:42.954955 13837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1bb181eac61946a58c33dcfdc076a5b2, State: Initialized, Role: LEARNER
I20260807 06:33:42.955010 13837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } }
I20260807 06:33:42.955266 13837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:42.955240 13842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } }
I20260807 06:33:42.955319 13842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:33:42.957054 13865 catalog_manager.cc:1259] Loaded cluster ID: 42e49fac30c54cb99f4487985a373342
I20260807 06:33:42.957414 13865 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: loading cluster ID for follower catalog manager: success
I20260807 06:33:42.958308 13865 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: acquiring CA information for follower catalog manager: success
I20260807 06:33:42.958631 13865 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:33:42.964727 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:42.957113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.2.243.61:37469)
0807 06:33:42.957268 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:42.957270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:42.957280 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:42.957305 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:42.957306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:42.957318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:42.957399 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:42.958150 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.959252 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.960100 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:42.960338 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:42.960427 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:42.961797 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:42.961798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:42.961799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:42.961807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:42.963362 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.963810 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.963812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.963812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.963880 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:42.964117 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:42.964118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:42.964118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:42.964242 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:42.964359 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:42.964479 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 06:33:42.964613 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.2.243.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.60.keytab.
Entry for principal oryx/127.2.243.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.60.keytab.
I20260807 06:33:42.995095 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:37239
--webserver_interface=127.2.243.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.60.keytab
--principal=oryx/127.2.243.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:43.080034 13871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:43.080235 13871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:43.080257 13871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:43.081452 13871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:43.081494 13871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:43.081509 13871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:43.081521 13871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:43.081533 13871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:43.083038 13871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.60.keytab
--principal=oryx/127.2.243.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:37239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.243.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:43.083312 13871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:43.083547 13871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:43.088603 13885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.088605 13883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:43.088676 13871 server_base.cc:1034] running on GCE node
W20260807 06:33:43.088649 13881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:43.089146 13871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:43.089433 13871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:43.090596 13871 hybrid_clock.cc:648] HybridClock initialized: now 1786084423090585 us; error 28 us; skew 500 ppm
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:43.093134 13871 init.cc:377] Logged in from keytab as oryx/127.2.243.60@KRBTEST.COM (short username oryx)
I20260807 06:33:43.093626 13871 webserver.cc:460] Webserver started at http://127.2.243.60:36097/ using document root <none> and password file <none>
I20260807 06:33:43.093816 13871 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:43.093858 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:43.093942 13871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:43.094887 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/instance:
uuid: "dc93dcb614524497b8185097e9d8b12c"
format_stamp: "Formatted at 2026-08-07 06:33:43 on dist-test-slave-rwrg"
server_key: "ea63aac3af4e19e562be1ef0f64d2444"
server_key_iv: "40477b0b660b4bac9946e14b78e9a3cf"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.095187 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal/instance:
uuid: "dc93dcb614524497b8185097e9d8b12c"
format_stamp: "Formatted at 2026-08-07 06:33:43 on dist-test-slave-rwrg"
server_key: "ea63aac3af4e19e562be1ef0f64d2444"
server_key_iv: "40477b0b660b4bac9946e14b78e9a3cf"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.096424 13871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:33:43.097218 13897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.097427 13871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:43.097548 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
uuid: "dc93dcb614524497b8185097e9d8b12c"
format_stamp: "Formatted at 2026-08-07 06:33:43 on dist-test-slave-rwrg"
server_key: "ea63aac3af4e19e562be1ef0f64d2444"
server_key_iv: "40477b0b660b4bac9946e14b78e9a3cf"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.097641 13871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:43.116629 13871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:43.118737 13871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:43.118912 13871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:43.123416 13871 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.60:37239
I20260807 06:33:43.123504 13984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.60:37239 every 8 connection(s)
I20260807 06:33:43.123849 13871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
I20260807 06:33:43.125047 13987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:43.126614 13987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:33:43.130645 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 13871
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.60@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.243.60@KRBTEST.COM
I20260807 06:33:43.138073 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.127511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.2.243.62:42593)
0807 06:33:43.127589 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.127591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.127666 (+ 75us) server_negotiation.cc:407] Connection header received
0807 06:33:43.128224 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.128225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.128237 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.128296 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.129101 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.129634 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.130825 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.131051 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.131161 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.134072 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.134073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.134074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.134083 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.136220 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.136774 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.136776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.136776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.136855 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.137235 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.137236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.137236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.137378 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.137483 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.137858 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.137954 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:33:43.138278 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.127287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58842)
0807 06:33:43.127526 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.127589 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.128102 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.128386 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.128389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.128766 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.128996 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.129001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.129735 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.129736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.130682 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.130685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.131220 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.131221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.131409 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.132206 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.132218 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.133951 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.136306 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.136312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.136314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.136682 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.136970 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.136972 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.136972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.137137 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.137462 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.137469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.137654 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.137781 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.137995 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
I20260807 06:33:43.140465 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.131245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.2.243.60:37239)
0807 06:33:43.131428 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.131433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.131458 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:33:43.131477 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.131478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.131493 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.131560 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.132434 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.133189 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.133971 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.134270 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.134351 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.137320 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.137328 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.137328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.137335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.139220 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.139609 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.139612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.139612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.139679 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.139890 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.139891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.139891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.140041 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.140184 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.140276 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.140353 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":126,"threads_started":1}
I20260807 06:33:43.140880 13987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.60@KRBTEST.COM for oryx/127.2.243.61@KRBTEST.COM
I20260807 06:33:43.147753 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.141268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.0.0.1:53360)
0807 06:33:43.141308 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.141325 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.141375 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.141711 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.141712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.141849 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.141904 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.141905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.142577 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.142578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.143065 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.143066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.143400 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.143401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.143452 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.143841 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.143845 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.145170 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.146865 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.146866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.146867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.147022 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.147197 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.147198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.147198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.147246 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.147539 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.147539 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.147574 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.147627 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.147694 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:33:43.147876 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.141512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.2.243.61:37469)
0807 06:33:43.141585 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.141586 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.141593 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:33:43.141614 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.141615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.141624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.141668 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.141999 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.142480 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.143142 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.143332 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.143386 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.145263 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.145264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.145265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.145272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.146781 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.147089 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.147089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.147090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.147139 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.147333 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.147334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.147334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.147464 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.147541 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.147656 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.147740 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:33:43.148324 13987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.60@KRBTEST.COM for oryx/127.2.243.60@KRBTEST.COM
I20260807 06:33:43.154784 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.148622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:39694)
0807 06:33:43.148678 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.148689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.148728 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.148861 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.148862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.149040 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.149069 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.149070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.150033 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.150034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.150471 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.150472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.150865 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.150866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.150912 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.151362 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.151373 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.152662 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.153944 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.153944 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.153945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.154112 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.154281 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.154282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.154282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.154331 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.154498 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.154498 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.154531 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.154580 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.154687 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 06:33:43.154906 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.148699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.2.243.60:37239)
0807 06:33:43.148753 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.148755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.148763 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:43.148783 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.148783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.148800 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.148858 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.149166 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.149958 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.150569 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.150792 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.150857 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.152740 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.152741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.152742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.152748 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.153870 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.154177 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.154177 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.154178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.154225 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.154368 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.154368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.154368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.154450 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.154492 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.154599 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.154790 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":35}
I20260807 06:33:43.156453 13987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Bootstrap starting.
I20260807 06:33:43.157439 13987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:43.157723 13987 log.cc:824] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:43.158468 13987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: No bootstrap required, opened a new log
I20260807 06:33:43.159868 13987 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } }
I20260807 06:33:43.160007 13987 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:43.160065 13987 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc93dcb614524497b8185097e9d8b12c, State: Initialized, Role: FOLLOWER
I20260807 06:33:43.160209 13987 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } }
I20260807 06:33:43.160519 13994 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } } }
I20260807 06:33:43.160576 13987 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:43.160622 13994 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:33:43.162302 14023 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:33:43.162480 14023 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:33:43.164726 13871 master_runner.cc:428] Detected that this master dc93dcb614524497b8185097e9d8b12c is joining an existing cluster
I20260807 06:33:43.164781 13871 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.243.60:37239
I20260807 06:33:43.172812 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.165970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.2.243.62:42593)
0807 06:33:43.166022 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.166024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.166045 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:33:43.166133 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.166135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.166149 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.166211 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.166661 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.167436 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.168599 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.168891 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.168935 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.170113 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.170114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.170115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.170121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.171544 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.172061 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.172061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.172062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.172114 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.172340 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.172340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.172340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.172435 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.172483 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.172692 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.172749 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:33:43.172886 14042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.165797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58856)
0807 06:33:43.165981 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.165992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.166050 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.166259 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.166260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.166455 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.166563 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.166565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.167547 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.167548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.168450 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.168453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.169023 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.169024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.169087 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.169572 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.169577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.170012 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.171662 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.171662 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.171663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.171956 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.172180 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.172181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.172181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.172265 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.172522 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.172523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.172591 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.172674 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.172782 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084422, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.62@KRBTEST.COM for oryx/127.2.243.60@KRBTEST.COM
I20260807 06:33:43.192045 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.174950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:39710)
0807 06:33:43.174999 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.175009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.175061 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.175296 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.175297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.175434 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.175468 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.175469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.176128 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.176129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.176678 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.176680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.177044 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.177045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.177110 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.177491 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.177496 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.178893 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.190873 (+ 11980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.190874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.190874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.191131 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.191362 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.191363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.191363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.191424 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.191730 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.191740 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.191784 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.191829 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.191936 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 06:33:43.192104 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 12918
I20260807 06:33:43.192531 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.175019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.2.243.60:37239)
0807 06:33:43.175174 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.175176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.175181 (+ 5us) server_negotiation.cc:407] Connection header received
0807 06:33:43.175208 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.175209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.175217 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.175293 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.175550 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.176026 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.176754 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.176968 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.177053 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.178983 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.178985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.178985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.178992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.190714 (+ 11722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.191222 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.191223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.191223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.191280 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.191500 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.191500 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.191501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.191648 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.192184 (+ 536us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.192227 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.192316 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260807 06:33:43.196007 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.165842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.2.243.61:37469)
0807 06:33:43.165914 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.165917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.166356 (+ 439us) server_negotiation.cc:407] Connection header received
0807 06:33:43.166451 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.166453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.166465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.166495 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.166979 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.167704 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.168763 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.169056 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.169093 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.190486 (+ 21393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.190488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.190489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.190498 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.192229 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.195208 (+ 2979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.195209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.195209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.195267 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.195453 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.195454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.195455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.195593 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.195645 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.195832 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.195915 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:33:43.196329 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.165915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.0.0.1:53366)
0807 06:33:43.166292 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.166303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.166381 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.166571 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.166572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.166814 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.166887 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.166888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.167794 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.167795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.168665 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.168667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.169160 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.169160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.169244 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.189745 (+ 20501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.189749 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.190334 (+ 585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.194894 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.194895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.194895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.195112 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.195337 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.195337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.195338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.195388 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.195678 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.195678 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.195720 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.196122 (+ 402us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.196226 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":7,"thread_start_us":66,"threads_started":1}
I20260807 06:33:43.201840 12822 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.243.60:37239: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "dc93dcb614524497b8185097e9d8b12c"
member_type: NON_VOTER
last_known_addr {
host: "127.2.243.60"
port: 37239
}
attrs {
promote: true
}
}
dest_uuid: "b9e3c8301267400e901954e239974667"
cas_config_opid_index: 5
I20260807 06:33:43.202132 12822 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } }
I20260807 06:33:43.202497 13717 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER dc93dcb614524497b8185097e9d8b12c (127.2.243.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.203229 13941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:43.203681 13717 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.243.60:37239
I20260807 06:33:43.203763 13717 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.203840 13717 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:43.204319 13941 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Refusing update from remote peer b9e3c8301267400e901954e239974667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 06:33:43.204447 13871 master.cc:537] Master@127.2.243.60:37239 shutting down...
I20260807 06:33:43.204615 13718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:33:43.205390 12795 consensus_peers.cc:487] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 -> Peer dc93dcb614524497b8185097e9d8b12c (127.2.243.60:37239): Couldn't send request to peer dc93dcb614524497b8185097e9d8b12c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 06:33:43.207207 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:40539
--local_ip_for_outbound_sockets=127.2.243.1
--tserver_master_addrs=127.2.243.62:42593
--webserver_port=40419
--webserver_interface=127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab
--principal=oryx/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:33:43.207410 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.202662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.0.0.1:53370)
0807 06:33:43.203218 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.203231 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.203290 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.203599 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.203600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.203602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:43.203648 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.203650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.204843 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.204844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.206291 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.206294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.207150 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.207151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.207218 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.207239 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.207290 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.207329 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535}
I20260807 06:33:43.208002 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.202963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53370 (local address 127.2.243.61:37469)
0807 06:33:43.203407 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.203409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.203417 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:43.203437 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.203438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.203463 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.203524 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:43.203721 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.204483 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.206387 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.206977 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.207044 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.207114 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.207897 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.207939 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421}
I20260807 06:33:43.208240 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.166178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:39702)
0807 06:33:43.166311 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.166322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.166381 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.166539 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.166539 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.166813 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.166887 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.166888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.167765 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.167765 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.168665 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.168667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.169159 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.169159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.169245 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.188968 (+ 19723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.188973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.189505 (+ 532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.205582 (+ 16077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.205583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.205584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.205828 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.207569 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.207569 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.207570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.207629 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.207900 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.207901 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.207935 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.207982 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.208095 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":32,"thread_start_us":79,"threads_started":1}
I20260807 06:33:43.208647 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.166189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.2.243.60:37239)
0807 06:33:43.166224 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.166226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.166360 (+ 134us) server_negotiation.cc:407] Connection header received
0807 06:33:43.166448 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.166448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.166456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.166484 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.166970 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.167676 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.168757 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.169046 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.169088 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.189655 (+ 20567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.189656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.189657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.189664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.201720 (+ 12056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.207423 (+ 5703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.207425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.207425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.207486 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.207700 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.207701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.207702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.207827 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.207881 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.208459 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.208570 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:43.209100 13804 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 LEARNER]: Refusing update from remote peer b9e3c8301267400e901954e239974667: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 06:33:43.209339 13718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 06:33:43.209517 13871 raft_consensus.cc:2229] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:43.209615 13871 raft_consensus.cc:2258] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:43.209659 13871 tablet_replica.cc:330] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: stopping tablet replica
I20260807 06:33:43.209905 13804 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER dc93dcb614524497b8185097e9d8b12c (127.2.243.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.211200 13842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e3c8301267400e901954e239974667. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.211288 13842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:33:43.211323 13842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: NON_VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: true } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.211362 13842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:33:43.211767 13718 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: attempting to promote NON_VOTER 1bb181eac61946a58c33dcfdc076a5b2 to VOTER
I20260807 06:33:43.211995 13718 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } }
I20260807 06:33:43.212323 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.212196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:39722)
0807 06:33:43.212255 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.212293 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 06:33:43.212558 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.212332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:39726)
0807 06:33:43.212475 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.212533 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 06:33:43.212560 13804 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 LEARNER]: Refusing update from remote peer b9e3c8301267400e901954e239974667: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 06:33:43.212790 13717 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 06:33:43.212920 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.212815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:39732)
0807 06:33:43.212853 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.212890 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 06:33:43.213027 12795 consensus_peers.cc:487] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 -> Peer dc93dcb614524497b8185097e9d8b12c (127.2.243.60:37239): Couldn't send request to peer dc93dcb614524497b8185097e9d8b12c. Status: Network error: Client connection negotiation failed: client connection to 127.2.243.60:37239: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 06:33:43.213575 13719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.213798 13804 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.214663 13719 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.214708 13718 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.214776 13718 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:43.214742 13719 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:43.214955 14052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.215077 14052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:33:43.222877 13871 master.cc:552] Master@127.2.243.60:37239 shutdown complete.
I20260807 06:33:43.222929 13871 master_runner.cc:305] Clearing existing system tablet
I20260807 06:33:43.223076 13871 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:43.223124 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:43.223855 13871 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.224372 14055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.224483 13871 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:43.224556 13871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.224581 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
uuid: "dc93dcb614524497b8185097e9d8b12c"
format_stamp: "Formatted at 2026-08-07 06:33:43 on dist-test-slave-rwrg"
server_key: "ea63aac3af4e19e562be1ef0f64d2444"
server_key_iv: "40477b0b660b4bac9946e14b78e9a3cf"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.225334 13871 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:43.226850 13871 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 06:33:43.226920 13871 log.cc:1196] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 06:33:43.227121 13871 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Deleting consensus metadata
I20260807 06:33:43.227249 13871 master_runner.cc:315] Copying system tablet from 127.2.243.62:42593
I20260807 06:33:43.227864 13871 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: tablet copy: Beginning tablet copy session from remote peer at address 127.2.243.62:42593
I20260807 06:33:43.239887 14063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.228916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58860)
0807 06:33:43.232614 (+ 3698us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.232628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.232708 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.232992 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.232993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.233185 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.233243 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.233245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.234315 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.234316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.235147 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.235150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.235652 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.235653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.235715 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.236260 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.236265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.236758 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.238711 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.238712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.238713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.238938 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.239189 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.239190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.239190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.239258 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.239559 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.239559 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.239610 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.239681 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.239784 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3672,"thread_start_us":73,"threads_started":1}
I20260807 06:33:43.239905 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.228979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58860 (local address 127.2.243.62:42593)
0807 06:33:43.229023 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.229026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.232716 (+ 3690us) server_negotiation.cc:407] Connection header received
0807 06:33:43.232758 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.232759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.232772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.232851 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.233348 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.234202 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.235241 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.235532 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.235609 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.236845 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.236846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.236846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.236854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.238626 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.239046 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.239047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.239047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.239113 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.239328 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.239329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.239329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.239471 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.239550 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.239729 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.239813 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:33:43.240715 12854 tablet_copy_service.cc:140] P b9e3c8301267400e901954e239974667: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer dc93dcb614524497b8185097e9d8b12c ({username='oryx', principal='oryx/127.2.243.60@KRBTEST.COM'} at 127.0.0.1:58860)
I20260807 06:33:43.240779 12854 tablet_copy_service.cc:161] P b9e3c8301267400e901954e239974667: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer dc93dcb614524497b8185097e9d8b12c at {username='oryx', principal='oryx/127.2.243.60@KRBTEST.COM'} at 127.0.0.1:58860: session id = dc93dcb614524497b8185097e9d8b12c-00000000000000000000000000000000
I20260807 06:33:43.241341 12854 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 06:33:43.241861 13871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 06:33:43.243130 13871 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: tablet copy: Starting download of 0 data blocks...
I20260807 06:33:43.243319 13871 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: tablet copy: Starting download of 1 WAL segments...
I20260807 06:33:43.244045 13871 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 06:33:43.245018 12854 tablet_copy_service.cc:342] P b9e3c8301267400e901954e239974667: Request end of tablet copy session dc93dcb614524497b8185097e9d8b12c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.243.60@KRBTEST.COM'} at 127.0.0.1:58860
I20260807 06:33:43.245065 12854 tablet_copy_service.cc:433] P b9e3c8301267400e901954e239974667: ending tablet copy session dc93dcb614524497b8185097e9d8b12c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer dc93dcb614524497b8185097e9d8b12c
I20260807 06:33:43.247762 13871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:43.248874 14064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.249734 14065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:43.250021 13871 server_base.cc:1034] running on GCE node
W20260807 06:33:43.250034 14070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:43.250293 13871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:43.250451 13871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:43.251590 13871 hybrid_clock.cc:648] HybridClock initialized: now 1786084423251596 us; error 42 us; skew 500 ppm
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:43.253098 13871 init.cc:377] Logged in from keytab as oryx/127.2.243.60@KRBTEST.COM (short username oryx)
I20260807 06:33:43.253381 13871 webserver.cc:460] Webserver started at http://127.2.243.60:43933/ using document root <none> and password file <none>
I20260807 06:33:43.253510 13871 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:43.253549 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:43.254339 13871 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:43.255004 14080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.255200 13871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.255255 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
uuid: "dc93dcb614524497b8185097e9d8b12c"
format_stamp: "Formatted at 2026-08-07 06:33:43 on dist-test-slave-rwrg"
server_key: "ea63aac3af4e19e562be1ef0f64d2444"
server_key_iv: "40477b0b660b4bac9946e14b78e9a3cf"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.255311 13871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:43.263530 13871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:43.265374 13871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:43.268656 13871 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.60:37239
I20260807 06:33:43.269044 13871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
I20260807 06:33:43.269971 14171 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:33:43.270490 14171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Bootstrap starting.
I20260807 06:33:43.273547 14171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:33:43.273773 14171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Bootstrap complete.
I20260807 06:33:43.273986 14171 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } }
I20260807 06:33:43.274080 14171 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dc93dcb614524497b8185097e9d8b12c, State: Initialized, Role: LEARNER
I20260807 06:33:43.274118 14171 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } }
I20260807 06:33:43.274290 14171 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:43.274317 14174 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.274531 14174 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: This master's current role is: LEARNER
I20260807 06:33:43.275730 14188 catalog_manager.cc:1259] Loaded cluster ID: 42e49fac30c54cb99f4487985a373342
I20260807 06:33:43.275812 14188 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: loading cluster ID for follower catalog manager: success
I20260807 06:33:43.277124 14188 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: acquiring CA information for follower catalog manager: success
I20260807 06:33:43.277418 14188 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:33:43.280606 14170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.60:37239 every 8 connection(s)
W20260807 06:33:43.306332 14049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:43.306514 14049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:43.306555 14049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:43.307673 14049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:43.307713 14049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:43.307739 14049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:43.309273 14049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab
--principal=oryx/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:40539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=40419
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:43.309525 14049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:43.309756 14049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:33:43.315770 14049 server_base.cc:1034] running on GCE node
W20260807 06:33:43.315748 14195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.315778 14197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.315754 14200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:43.316104 14049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:43.316354 14049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:43.317518 14049 hybrid_clock.cc:648] HybridClock initialized: now 1786084423317493 us; error 41 us; skew 500 ppm
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:43.319976 14049 init.cc:377] Logged in from keytab as oryx/127.2.243.1@KRBTEST.COM (short username oryx)
I20260807 06:33:43.320382 14049 webserver.cc:460] Webserver started at http://127.2.243.1:40419/ using document root <none> and password file <none>
I20260807 06:33:43.320691 14049 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:43.320778 14049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:43.322193 14049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:43.322791 14208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.322952 14049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:43.323043 14049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "9be736c4bcf346bbb551433af3b5cb57"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "31ffa6178d11825acc444fc56a4c5c1d"
server_key_iv: "2bfe202b3a9d2acd3aac9f76b52a8bb7"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.323393 14049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:43.344027 14049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:43.346381 14049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:43.346551 14049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:43.346840 14049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:43.347218 14049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:43.347277 14049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.347352 14049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:43.347395 14049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.353912 14049 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40539
I20260807 06:33:43.353958 14397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40539 every 8 connection(s)
I20260807 06:33:43.354353 14049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:43.356070 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 14049
I20260807 06:33:43.356248 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 13139
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.61@KRBTEST.COM
I20260807 06:33:43.368331 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.355515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:45693 (local address 127.2.243.61:37469)
0807 06:33:43.355637 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.355639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.355691 (+ 52us) server_negotiation.cc:407] Connection header received
0807 06:33:43.357555 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.357556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.357568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.357596 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.358474 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.359113 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.360430 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.360867 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.360965 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.363807 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.363809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.363809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.363818 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.366159 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.366848 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.366849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.366850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.366925 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.367300 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.367301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.367301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.367490 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.367562 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.368063 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.368174 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 06:33:43.369120 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.355175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.2.243.1:45693)
0807 06:33:43.355545 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.355591 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.357466 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.357739 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.357743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.358127 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.358379 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.358384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.359226 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.359227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.360331 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.360335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.360968 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.360969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.361113 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.361723 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.361733 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.363675 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.366279 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.366289 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.366291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.366734 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.367016 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.367017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.367018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.367200 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.367625 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.367632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.367854 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.368466 (+ 612us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.368776 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":106,"threads_started":1}
I20260807 06:33:43.373353 14401 heartbeater.cc:347] Connected to a master server at 127.2.243.61:37469
I20260807 06:33:43.373477 14401 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.373735 14401 heartbeater.cc:511] Master 127.2.243.61:37469 requested a full tablet report, sending...
I20260807 06:33:43.374308 13788 ts_manager.cc:194] Registered new tserver with Master: 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1:40539)
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:43.382486 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:38873
--local_ip_for_outbound_sockets=127.2.243.2
--tserver_master_addrs=127.2.243.62:42593
--webserver_port=37437
--webserver_interface=127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab
--principal=oryx/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:33:43.382936 14407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.355175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:37915)
0807 06:33:43.355727 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.355774 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.360419 (+ 4645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.374432 (+ 14013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.374434 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.374660 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.374736 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.374739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.375691 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.375692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.376873 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.376877 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.377497 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.377498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.377572 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.378067 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.378073 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.379785 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.381696 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.381697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.381698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.381896 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.382162 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.382163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.382164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.382244 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.382572 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.382573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.382652 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.382725 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.382834 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260807 06:33:43.383252 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.355579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:37915 (local address 127.2.243.62:42593)
0807 06:33:43.355653 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.355655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.355827 (+ 172us) server_negotiation.cc:407] Connection header received
0807 06:33:43.360527 (+ 4700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.360529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.360543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.360595 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.374848 (+ 14253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.375576 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.377013 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.377362 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.377410 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.379887 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.379889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.379890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.379906 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.381617 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.381997 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.381998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.381999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.382068 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.382332 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.382333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.382333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.382484 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.382552 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.383093 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.383181 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:33:43.383723 14398 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42593
I20260807 06:33:43.383803 14398 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.383886 14398 heartbeater.cc:511] Master 127.2.243.62:42593 requested a full tablet report, sending...
I20260807 06:33:43.384209 12822 ts_manager.cc:194] Re-registered known tserver with Master: 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1:40539)
I20260807 06:33:43.384644 12822 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:37915
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.60@KRBTEST.COM
I20260807 06:33:43.388521 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.357469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:33431 (local address 127.2.243.60:37239)
0807 06:33:43.377899 (+ 20430us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.377901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.377912 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:43.377966 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.377967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.377981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.378808 (+ 827us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.379275 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.380379 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.381468 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.381838 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.381916 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.385735 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.385736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.385737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.385746 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.387315 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.387688 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.387689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.387690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.387747 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.387950 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.387950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.387951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.388067 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.388146 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.388317 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.388398 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20401,"thread_start_us":1645,"threads_started":1}
I20260807 06:33:43.388608 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.355175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.2.243.1:33431)
0807 06:33:43.355824 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.355834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.357342 (+ 1508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.378857 (+ 21515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.378858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.379082 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.379160 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.379163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.380482 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.380483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.381360 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.381363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.382033 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.382035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.382112 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.383430 (+ 1318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.383435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.385646 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.387425 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.387426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.387427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.387607 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.387820 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.387821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.387821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.387881 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.388137 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.388138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.388196 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.388277 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.388392 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":283,"thread_start_us":83,"threads_started":1}
I20260807 06:33:43.389012 14400 heartbeater.cc:347] Connected to a master server at 127.2.243.60:37239
I20260807 06:33:43.389068 14400 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.389161 14400 heartbeater.cc:511] Master 127.2.243.60:37239 requested a full tablet report, sending...
I20260807 06:33:43.389569 14091 ts_manager.cc:194] Registered new tserver with Master: 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1:40539)
W20260807 06:33:43.469755 14410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:43.470014 14410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:43.470055 14410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:43.471230 14410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:43.471299 14410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:43.471345 14410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:43.472890 14410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab
--principal=oryx/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:38873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=37437
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:43.473181 14410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:43.473433 14410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:43.478772 14420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.478741 14417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.478848 14416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:43.479166 14410 server_base.cc:1034] running on GCE node
I20260807 06:33:43.479333 14410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:43.479570 14410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:43.480715 14410 hybrid_clock.cc:648] HybridClock initialized: now 1786084423480703 us; error 27 us; skew 500 ppm
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:43.483311 14410 init.cc:377] Logged in from keytab as oryx/127.2.243.2@KRBTEST.COM (short username oryx)
I20260807 06:33:43.483721 14410 webserver.cc:460] Webserver started at http://127.2.243.2:37437/ using document root <none> and password file <none>
I20260807 06:33:43.483984 14410 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:43.484074 14410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:43.485558 14410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:43.486195 14429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.486382 14410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:43.486492 14410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "7bfcc9cc7f452c3b8418cd1f9bbb1fce"
server_key_iv: "bece444551c58923f299befb83e8db06"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.486846 14410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:43.509915 14410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:43.512131 14410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:43.512296 14410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:43.512583 14410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:43.512905 14410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:43.512960 14410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.513011 14410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:43.513038 14410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.519109 14410 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:38873
I20260807 06:33:43.519157 14623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:38873 every 8 connection(s)
I20260807 06:33:43.519577 14410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.60@KRBTEST.COM
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.61@KRBTEST.COM
I20260807 06:33:43.529098 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 14410
I20260807 06:33:43.530110 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 13348
I20260807 06:33:43.531810 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.520753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:32769 (local address 127.2.243.62:42593)
0807 06:33:43.520811 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.520812 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.520822 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:43.521925 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.521927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.521942 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.521985 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.523631 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.524102 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525124 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.525356 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525421 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.528267 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.528268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.528268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.528276 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.530303 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.530791 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.530793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.530793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.530848 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.531083 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.531084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.531084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.531216 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.531253 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.531567 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.531682 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:33:43.532346 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.520375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:32769)
0807 06:33:43.520761 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.520771 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.521784 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.522086 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.522087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.522429 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.522645 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.522648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.524189 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.524190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.525038 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.525040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.525438 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.525572 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.526266 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.526276 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.528161 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.530394 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.530401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.530404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.530713 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.530910 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.530911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.530912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.531026 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.531291 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.531295 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.531446 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.531912 (+ 466us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.532092 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":7,"thread_start_us":113,"threads_started":1}
I20260807 06:33:43.540539 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.520733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:36681 (local address 127.2.243.60:37239)
0807 06:33:43.520775 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.520777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.520988 (+ 211us) server_negotiation.cc:407] Connection header received
0807 06:33:43.521942 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.521943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.521955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.521986 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.522727 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.523511 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.524970 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.525298 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525360 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.528901 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.528903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.528903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.528911 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.530742 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.539374 (+ 8632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.539376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.539377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.539469 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.539805 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.539806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.539806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.539988 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.540063 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.540274 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.540381 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:33:43.540946 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.520375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.2.243.2:36681)
0807 06:33:43.520906 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.520916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.521780 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.522089 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.522089 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.522432 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.522645 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.522648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.523612 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.523613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.524876 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.524879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.525435 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.525572 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.526345 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.526349 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.528761 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.539005 (+ 10244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.539007 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.539008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.539256 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.539588 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.539590 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.539591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.539700 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.540105 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.540106 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.540154 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.540654 (+ 500us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.540774 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":123,"thread_start_us":137,"threads_started":1}
I20260807 06:33:43.541296 14626 heartbeater.cc:347] Connected to a master server at 127.2.243.60:37239
I20260807 06:33:43.541380 14626 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.541571 14626 heartbeater.cc:511] Master 127.2.243.60:37239 requested a full tablet report, sending...
I20260807 06:33:43.541967 14091 ts_manager.cc:194] Registered new tserver with Master: 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2:38873)
I20260807 06:33:43.545588 14624 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42593
I20260807 06:33:43.545680 14624 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.545774 14624 heartbeater.cc:511] Master 127.2.243.62:42593 requested a full tablet report, sending...
I20260807 06:33:43.546063 12822 ts_manager.cc:194] Re-registered known tserver with Master: 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2:38873)
I20260807 06:33:43.546471 12822 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:32769
I20260807 06:33:43.548605 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.520595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:48479 (local address 127.2.243.61:37469)
0807 06:33:43.520661 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.520664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.520750 (+ 86us) server_negotiation.cc:407] Connection header received
0807 06:33:43.521859 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.521860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.521883 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.521931 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.522658 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.523509 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525328 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.525594 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525676 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.539437 (+ 13761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.539439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.539439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.539448 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.547173 (+ 7725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.547535 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.547537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.547537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.547608 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.547880 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.547881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.547881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.548030 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.548103 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.548329 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.548448 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 06:33:43.548832 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.520374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.2.243.2:48479)
0807 06:33:43.520600 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.520645 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.521742 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.522015 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.522018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.522360 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.522561 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.522565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.524576 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.524577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.525240 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.525243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.525689 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.525690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.525775 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.526544 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.526548 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.529349 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.547261 (+ 17912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.547262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.547263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.547443 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.547664 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.547664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.547664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.547708 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.548126 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.548127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.548190 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.548268 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.548449 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":141,"threads_started":1}
I20260807 06:33:43.549299 14629 heartbeater.cc:347] Connected to a master server at 127.2.243.61:37469
I20260807 06:33:43.549363 14629 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.549446 14629 heartbeater.cc:511] Master 127.2.243.61:37469 requested a full tablet report, sending...
I20260807 06:33:43.549853 13788 ts_manager.cc:194] Registered new tserver with Master: 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2:38873)
I20260807 06:33:43.551290 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:38203
--local_ip_for_outbound_sockets=127.2.243.3
--tserver_master_addrs=127.2.243.62:42593
--webserver_port=42531
--webserver_interface=127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab
--principal=oryx/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:43.635927 14634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:43.636251 14634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:43.636298 14634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:43.638150 14634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:43.638206 14634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:43.638235 14634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:43.639845 14634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab
--principal=oryx/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:38203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=42531
--tserver_master_addrs=127.2.243.62:42593,127.2.243.61:37469,127.2.243.60:37239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:43.640064 14634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:43.640244 14634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:43.645699 14642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.645699 14641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:43.645713 14644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:43.645978 14634 server_base.cc:1034] running on GCE node
I20260807 06:33:43.646215 14634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:43.646471 14634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:43.647629 14634 hybrid_clock.cc:648] HybridClock initialized: now 1786084423647608 us; error 43 us; skew 500 ppm
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:43.650697 14634 init.cc:377] Logged in from keytab as oryx/127.2.243.3@KRBTEST.COM (short username oryx)
I20260807 06:33:43.651151 14634 webserver.cc:460] Webserver started at http://127.2.243.3:42531/ using document root <none> and password file <none>
I20260807 06:33:43.651379 14634 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:43.651454 14634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:43.652940 14634 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:43.653716 14656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.653942 14634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:43.654047 14634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "48b2a453116349bca087486129d966f8"
format_stamp: "Formatted at 2026-08-07 06:33:42 on dist-test-slave-rwrg"
server_key: "5c45667bac63bb21e0137f676bc14610"
server_key_iv: "e9cc247952c1e6e1264caad9419aa482"
server_key_version: "encryptionkey@0"
I20260807 06:33:43.654393 14634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:43.664935 14634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:43.667443 14634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:43.667622 14634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:43.667876 14634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:43.668221 14634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:43.668278 14634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.668324 14634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:43.668354 14634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:43.675036 14634 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:38203
I20260807 06:33:43.675114 14859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:38203 every 8 connection(s)
I20260807 06:33:43.675477 14634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:43.676846 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 14634
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.60@KRBTEST.COM
I20260807 06:33:43.687764 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.678106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.2.243.61:37469)
0807 06:33:43.678157 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.678159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.678534 (+ 375us) server_negotiation.cc:407] Connection header received
0807 06:33:43.678577 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.678578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.678591 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.678673 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.679540 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.680266 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.682285 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.682594 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.682635 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.684579 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.684580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.684581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.684589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.686274 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.686827 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.686828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.686829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.686890 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.687108 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.687109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.687109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.687242 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.687313 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.687571 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.687642 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:33:43.688139 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.678308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39734 (local address 127.2.243.60:37239)
0807 06:33:43.678423 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.678426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.680279 (+ 1853us) server_negotiation.cc:407] Connection header received
0807 06:33:43.680306 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.680307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.680322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.680379 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.681169 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.681931 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.682914 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.683128 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.683205 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.684650 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.684652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.684652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.684658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.686464 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.686928 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.686929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.686929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.686986 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.687195 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.687195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.687196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.687310 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.687397 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.687778 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.687862 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260807 06:33:43.691498 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.676572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:46203 (local address 127.2.243.60:37239)
0807 06:33:43.676646 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.676648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.676663 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:33:43.680674 (+ 4011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.680675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.680687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.680739 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.681118 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.681880 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.683378 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.683936 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.683987 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.688147 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.688148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.688149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.688156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.689756 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690339 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.690340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.690340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.690393 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690740 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.690741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.690741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.690880 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.690937 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.691342 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.691419 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:33:43.691573 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.677895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.2.243.62:42593)
0807 06:33:43.678738 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.678740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.678863 (+ 123us) server_negotiation.cc:407] Connection header received
0807 06:33:43.678913 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.678914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.678928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.678997 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.679600 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.680321 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.682874 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.685839 (+ 2965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.685900 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.687509 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.687510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.687510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.687517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.690134 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690519 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.690521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.690521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.690587 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690780 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.690780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.690781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.690902 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.690985 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.691390 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.691475 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":107,"threads_started":1}
I20260807 06:33:43.691964 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.676312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:55147 (local address 127.2.243.62:42593)
0807 06:33:43.676651 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.676653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.676666 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:33:43.679094 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.679095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.679104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.679141 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.679900 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.680582 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.682109 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.682402 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.682462 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.685501 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.685502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.685503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.685511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.690175 (+ 4664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690584 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.690585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.690585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.690644 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690844 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.690845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.690845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.690973 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.691030 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.691800 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.691875 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":299}
I20260807 06:33:43.692221 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.676333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:55147)
0807 06:33:43.676541 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.676586 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.679015 (+ 2429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.679225 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.679234 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.679542 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.679800 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.679806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.680666 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.680667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.682014 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.682016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.682522 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.682523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.682653 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.683178 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.683234 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.685398 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.690264 (+ 4866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.690266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.690496 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.690712 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.690713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.690775 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.691053 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.691054 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.691147 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.691257 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.691425 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1,"thread_start_us":118,"threads_started":1}
I20260807 06:33:43.692221 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.676333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.2.243.3:46203)
0807 06:33:43.676579 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.676592 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.680088 (+ 3496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.680783 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.680784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.680954 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.681032 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.681034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.681972 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.681973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.682773 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.682775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.684069 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.684070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.684135 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.684618 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.684624 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.688048 (+ 3424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.689836 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.689843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.689846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.690253 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.690464 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.690465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.690466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.690621 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.690964 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.690971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.691147 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.691257 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.692006 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":625,"thread_start_us":102,"threads_started":1}
I20260807 06:33:43.692888 14861 heartbeater.cc:347] Connected to a master server at 127.2.243.60:37239
I20260807 06:33:43.692943 14860 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42593
I20260807 06:33:43.692994 14861 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.693004 14860 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.693194 14860 heartbeater.cc:511] Master 127.2.243.62:42593 requested a full tablet report, sending...
I20260807 06:33:43.693202 14861 heartbeater.cc:511] Master 127.2.243.60:37239 requested a full tablet report, sending...
I20260807 06:33:43.693609 12822 ts_manager.cc:194] Re-registered known tserver with Master: 48b2a453116349bca087486129d966f8 (127.2.243.3:38203)
I20260807 06:33:43.693707 14091 ts_manager.cc:194] Registered new tserver with Master: 48b2a453116349bca087486129d966f8 (127.2.243.3:38203)
I20260807 06:33:43.694087 12822 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:55147
Aug 07 06:33:43 dist-test-slave-rwrg krb5kdc[12737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084423, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.61@KRBTEST.COM
I20260807 06:33:43.693781 12821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:43.696246 12821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:43.699048 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.677936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.2.243.3:49455)
0807 06:33:43.678608 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.678616 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.680033 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.689978 (+ 9945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.689979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.690372 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:43.690448 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.690449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.691395 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.691397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.692136 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.692138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.693687 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.693688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.693753 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.694197 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:43.694201 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:43.695539 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:43.697531 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.697533 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.697533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.697765 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.698031 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:43.698032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:43.698032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:43.698110 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:43.698446 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:43.698447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:43.698496 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.698560 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.698676 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":79,"threads_started":1}
I20260807 06:33:43.699369 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.678246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:49455 (local address 127.2.243.61:37469)
0807 06:33:43.678934 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.678936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.678945 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:43.689680 (+ 10735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.689681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.689863 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.690209 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:43.690573 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.691298 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.692483 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.693590 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.693890 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.695664 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:43.695665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:43.695665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:43.695672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:43.697443 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.697868 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.697869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.697870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.697936 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:43.698189 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:43.698190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:43.698190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:43.698342 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:43.698402 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.699180 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.699279 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":67,"threads_started":1}
I20260807 06:33:43.699743 14862 heartbeater.cc:347] Connected to a master server at 127.2.243.61:37469
I20260807 06:33:43.699805 14862 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:43.699885 14862 heartbeater.cc:511] Master 127.2.243.61:37469 requested a full tablet report, sending...
I20260807 06:33:43.700222 13788 ts_manager.cc:194] Registered new tserver with Master: 48b2a453116349bca087486129d966f8 (127.2.243.3:38203)
I20260807 06:33:43.704723 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.700398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38203 (local address 127.0.0.1:47798)
0807 06:33:43.701226 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.701242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.701285 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.701608 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.701609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.701611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:43.701679 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.701681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.702663 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.702664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.703464 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.703467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.704438 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.704439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.704534 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.704540 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.704595 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.704644 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260807 06:33:43.704864 14900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.700715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:38873 (local address 127.0.0.1:59394)
0807 06:33:43.701342 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.701348 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.701385 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.701592 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.701593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.701595 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:43.701661 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.701663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.702936 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.702937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.703899 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.703901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.704657 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.704657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.704713 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.704717 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.704747 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.704782 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":22,"thread_start_us":75,"threads_started":1}
I20260807 06:33:43.705032 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.700326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40539 (local address 127.0.0.1:58732)
0807 06:33:43.700831 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.700843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.700902 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.701110 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.701111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.701113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:43.701155 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.701156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.702459 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.702460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.703570 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.703573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.704830 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.704831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.704896 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.704899 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.704926 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.704966 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":24}
I20260807 06:33:43.705235 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.701048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.2.243.3:38203)
0807 06:33:43.701461 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.701465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.701472 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:33:43.701498 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.701499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.701510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.701608 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:43.701751 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.702582 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.703577 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.704320 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.704430 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.705078 (+ 648us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.705108 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.705153 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":76,"threads_started":1}
I20260807 06:33:43.705243 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.700828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.2.243.2:38873)
0807 06:33:43.700999 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.701003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.701416 (+ 413us) server_negotiation.cc:407] Connection header received
0807 06:33:43.701463 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.701465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.701480 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.701718 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:43.701771 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.702850 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.703989 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.704568 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.704967 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.705079 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.705110 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.705163 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260807 06:33:43.705451 14897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.700470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.2.243.1:40539)
0807 06:33:43.700775 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.700780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.700893 (+ 113us) server_negotiation.cc:407] Connection header received
0807 06:33:43.700996 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.700998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.701017 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.701101 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:43.701207 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.702289 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.703644 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.704719 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.705206 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.705306 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.705335 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.705375 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":184,"threads_started":1}
I20260807 06:33:43.705909 14519 tablet_service.cc:1467] Processing CreateTablet for tablet 5c210e053daa4cfa93c6a2077faf704d (DEFAULT_TABLE table=test-table [id=b3178d215d6d41e59cac48b6c4e5cb39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:43.705928 14743 tablet_service.cc:1467] Processing CreateTablet for tablet 5c210e053daa4cfa93c6a2077faf704d (DEFAULT_TABLE table=test-table [id=b3178d215d6d41e59cac48b6c4e5cb39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:43.706094 14288 tablet_service.cc:1467] Processing CreateTablet for tablet 5c210e053daa4cfa93c6a2077faf704d (DEFAULT_TABLE table=test-table [id=b3178d215d6d41e59cac48b6c4e5cb39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:43.706300 14743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c210e053daa4cfa93c6a2077faf704d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:43.706298 14519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c210e053daa4cfa93c6a2077faf704d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:43.706374 14288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c210e053daa4cfa93c6a2077faf704d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:43.710428 14907 tablet_bootstrap.cc:492] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Bootstrap starting.
I20260807 06:33:43.710650 14904 tablet_bootstrap.cc:492] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Bootstrap starting.
I20260807 06:33:43.710794 14906 tablet_bootstrap.cc:492] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Bootstrap starting.
I20260807 06:33:43.711122 14907 tablet_bootstrap.cc:654] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:43.711447 14907 log.cc:824] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:43.711589 14906 tablet_bootstrap.cc:654] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:43.711905 14906 log.cc:824] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:43.712136 14904 tablet_bootstrap.cc:654] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:43.712574 14904 log.cc:824] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:43.712702 14906 tablet_bootstrap.cc:492] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: No bootstrap required, opened a new log
I20260807 06:33:43.712765 14906 ts_tablet_manager.cc:1397] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:43.713440 14907 tablet_bootstrap.cc:492] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: No bootstrap required, opened a new log
I20260807 06:33:43.713527 14907 ts_tablet_manager.cc:1397] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:33:43.714831 14904 tablet_bootstrap.cc:492] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: No bootstrap required, opened a new log
I20260807 06:33:43.714912 14904 ts_tablet_manager.cc:1397] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260807 06:33:43.715427 14906 raft_consensus.cc:348] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.715590 14906 raft_consensus.cc:374] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:43.715626 14906 raft_consensus.cc:729] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b2a453116349bca087486129d966f8, State: Initialized, Role: FOLLOWER
I20260807 06:33:43.715732 14907 raft_consensus.cc:348] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.715888 14907 raft_consensus.cc:374] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:43.715931 14907 raft_consensus.cc:729] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce79e330fa4470b8c5d6ac8ff0aff18, State: Initialized, Role: FOLLOWER
I20260807 06:33:43.715883 14906 consensus_queue.cc:260] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.716063 14907 consensus_queue.cc:260] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.716171 14906 ts_tablet_manager.cc:1428] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:33:43.716288 14860 heartbeater.cc:503] Master 127.2.243.62:42593 was elected leader, sending a full tablet report...
I20260807 06:33:43.716341 14907 ts_tablet_manager.cc:1428] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:33:43.716549 14624 heartbeater.cc:503] Master 127.2.243.62:42593 was elected leader, sending a full tablet report...
I20260807 06:33:43.717592 14904 raft_consensus.cc:348] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.717767 14904 raft_consensus.cc:374] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:43.717815 14904 raft_consensus.cc:729] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be736c4bcf346bbb551433af3b5cb57, State: Initialized, Role: FOLLOWER
I20260807 06:33:43.717993 14904 consensus_queue.cc:260] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.718266 14904 ts_tablet_manager.cc:1428] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260807 06:33:43.718410 14398 heartbeater.cc:503] Master 127.2.243.62:42593 was elected leader, sending a full tablet report...
W20260807 06:33:43.771302 14630 tablet.cc:2369] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:43.809510 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.806906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39746 (local address 127.2.243.60:37239)
0807 06:33:43.806975 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.806977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.806986 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:43.807021 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.807022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.807043 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.807125 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:43.807274 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.807772 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.808757 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.809150 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.809252 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.809344 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.809395 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.809431 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:33:43.809510 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.806747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:39746)
0807 06:33:43.806839 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.806852 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.806917 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.807132 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.807133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.807134 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:43.807201 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.807203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.807864 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.807866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.808666 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.808669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.809250 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.809251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.809323 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.809326 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.809379 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.809424 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 06:33:43.810015 14122 raft_consensus.cc:1203] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260807 06:33:43.811223 14920 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e3c8301267400e901954e239974667. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.811358 14920 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: This master's current role is: LEARNER
W20260807 06:33:43.856439 14402 tablet.cc:2369] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:33:43.930265 14863 tablet.cc:2369] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:43.985426 14917 raft_consensus.cc:482] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:43.985623 14917 raft_consensus.cc:504] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.986028 14917 leader_election.cc:288] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b2a453116349bca087486129d966f8 (127.2.243.3:38203), 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2:38873)
I20260807 06:33:43.990157 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.986276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:46139 (local address 127.2.243.3:38203)
0807 06:33:43.986329 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.986331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.986429 (+ 98us) server_negotiation.cc:407] Connection header received
0807 06:33:43.986459 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.986460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.986472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.986559 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:43.986833 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.987575 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.989166 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.989779 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.989851 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.989863 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.990039 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.990086 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:33:43.990159 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.986340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:35773 (local address 127.2.243.2:38873)
0807 06:33:43.986383 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:43.986385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:43.986396 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:43.986454 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:43.986455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:43.986467 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:43.986599 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:43.986829 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.987575 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.989166 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.989770 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.989852 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.989863 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:43.990064 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 06:33:43.990099 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:43.990157 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.986184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:38873 (local address 127.2.243.1:35773)
0807 06:33:43.986318 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.986330 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.986389 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.986626 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.986626 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.986630 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:43.986729 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.986732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.987675 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.987676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.989070 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.989073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.989866 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.989867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.989944 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.989947 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.990007 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.990055 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":17,"thread_start_us":55,"threads_started":1}
I20260807 06:33:43.990165 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:43.986191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38203 (local address 127.2.243.1:46139)
0807 06:33:43.986337 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:43.986348 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:43.986407 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:43.986625 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:43.986626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:43.986629 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:43.986729 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.986731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.987676 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.987677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.989070 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:43.989073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:43.989864 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:43.989866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:43.989938 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:43.989943 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:43.990007 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:33:43.990055 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":132,"thread_start_us":54,"threads_started":1}
I20260807 06:33:43.990608 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c210e053daa4cfa93c6a2077faf704d" candidate_uuid: "9be736c4bcf346bbb551433af3b5cb57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" is_pre_election: true
I20260807 06:33:43.990763 14555 raft_consensus.cc:2454] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be736c4bcf346bbb551433af3b5cb57 in term 0.
I20260807 06:33:43.990747 14771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c210e053daa4cfa93c6a2077faf704d" candidate_uuid: "9be736c4bcf346bbb551433af3b5cb57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b2a453116349bca087486129d966f8" is_pre_election: true
I20260807 06:33:43.990940 14771 raft_consensus.cc:2454] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be736c4bcf346bbb551433af3b5cb57 in term 0.
I20260807 06:33:43.991020 14210 leader_election.cc:302] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce79e330fa4470b8c5d6ac8ff0aff18, 9be736c4bcf346bbb551433af3b5cb57; no voters:
I20260807 06:33:43.991189 14917 raft_consensus.cc:2781] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:43.991273 14917 raft_consensus.cc:482] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:43.991310 14917 raft_consensus.cc:3037] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:43.992079 14917 raft_consensus.cc:504] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.992239 14917 leader_election.cc:288] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [CANDIDATE]: Term 1 election: Requested vote from peers 48b2a453116349bca087486129d966f8 (127.2.243.3:38203), 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2:38873)
I20260807 06:33:43.992476 14771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c210e053daa4cfa93c6a2077faf704d" candidate_uuid: "9be736c4bcf346bbb551433af3b5cb57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b2a453116349bca087486129d966f8"
I20260807 06:33:43.992619 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c210e053daa4cfa93c6a2077faf704d" candidate_uuid: "9be736c4bcf346bbb551433af3b5cb57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18"
I20260807 06:33:43.992636 14771 raft_consensus.cc:3037] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:43.992693 14555 raft_consensus.cc:3037] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:43.993639 14771 raft_consensus.cc:2454] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be736c4bcf346bbb551433af3b5cb57 in term 1.
I20260807 06:33:43.993880 14214 leader_election.cc:302] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b2a453116349bca087486129d966f8, 9be736c4bcf346bbb551433af3b5cb57; no voters:
I20260807 06:33:43.993989 14917 raft_consensus.cc:2781] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:43.994062 14555 raft_consensus.cc:2454] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be736c4bcf346bbb551433af3b5cb57 in term 1.
I20260807 06:33:43.994158 14917 raft_consensus.cc:686] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 1 LEADER]: Becoming Leader. State: Replica: 9be736c4bcf346bbb551433af3b5cb57, State: Running, Role: LEADER
I20260807 06:33:43.994282 14917 consensus_queue.cc:237] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } }
I20260807 06:33:43.995209 12821 catalog_manager.cc:5495] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1). New cstate: current_term: 1 leader_uuid: "9be736c4bcf346bbb551433af3b5cb57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } health_report { overall_health: UNKNOWN } } }
I20260807 06:33:43.995805 13717 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: NON_VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: true } } }
I20260807 06:33:43.995909 13717 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:43.996699 13719 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: attempting to promote NON_VOTER dc93dcb614524497b8185097e9d8b12c to VOTER
I20260807 06:33:43.996815 13719 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:43.997215 13804 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Refusing update from remote peer b9e3c8301267400e901954e239974667: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260807 06:33:43.997337 14120 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 LEARNER]: Refusing update from remote peer b9e3c8301267400e901954e239974667: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260807 06:33:43.997409 13718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260807 06:33:43.997520 13718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260807 06:33:43.998188 13719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, dc93dcb614524497b8185097e9d8b12c (127.2.243.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:43.998374 14120 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, dc93dcb614524497b8185097e9d8b12c (127.2.243.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:43.999068 13719 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:43.999130 13717 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:43.999187 13718 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:43.999229 13717 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:43.999253 13718 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:43.999238 14920 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:43.999322 14920 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:33:43.999150 13719 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:43.999531 13804 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, dc93dcb614524497b8185097e9d8b12c (127.2.243.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:44.000393 14935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9e3c8301267400e901954e239974667" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:44.000478 14935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:33:44.060515 14897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:44.057927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58738 (local address 127.2.243.1:40539)
0807 06:33:44.057986 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:44.057988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:44.057998 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:44.058043 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:44.058044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:44.058075 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:44.058155 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:44.058317 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.058877 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.059658 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.059856 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.059935 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:44.060122 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:44.060228 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:44.060317 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:44.060377 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:33:44.060400 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:33:44.062463 14555 raft_consensus.cc:1261] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 FOLLOWER]: Refusing update from remote peer 9be736c4bcf346bbb551433af3b5cb57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:44.062479 14771 raft_consensus.cc:1261] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Refusing update from remote peer 9be736c4bcf346bbb551433af3b5cb57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:44.062855 14917 consensus_queue.cc:1035] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:44.062995 14931 consensus_queue.cc:1035] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:44.065383 14938 mvcc.cc:205] Tried to move back new op lower bound from 7315801800956895232 to 7315801800681140224. Current Snapshot: MvccSnapshot[applied={T|T < 7315801800956895232}]
I20260807 06:33:44.065884 14939 mvcc.cc:205] Tried to move back new op lower bound from 7315801800956895232 to 7315801800681140224. Current Snapshot: MvccSnapshot[applied={T|T < 7315801800956895232}]
I20260807 06:33:44.068058 14941 mvcc.cc:205] Tried to move back new op lower bound from 7315801800956895232 to 7315801800681140224. Current Snapshot: MvccSnapshot[applied={T|T < 7315801800956895232 or (T in {7315801800956895232})}]
I20260807 06:33:44.068066 12822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:44.068161 12822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:44.069702 12822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9e3c8301267400e901954e239974667: Sending DeleteTablet for 3 replicas of tablet 5c210e053daa4cfa93c6a2077faf704d
I20260807 06:33:44.070016 14288 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c210e053daa4cfa93c6a2077faf704d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:44 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:58732
I20260807 06:33:44.070087 14519 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c210e053daa4cfa93c6a2077faf704d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:44 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:59394
I20260807 06:33:44.070187 14743 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c210e053daa4cfa93c6a2077faf704d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:44 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:47798
I20260807 06:33:44.070261 14951 tablet_replica.cc:330] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: stopping tablet replica
I20260807 06:33:44.070358 14951 raft_consensus.cc:2229] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:44.070426 14953 tablet_replica.cc:330] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: stopping tablet replica
I20260807 06:33:44.070457 14951 raft_consensus.cc:2258] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:44.070515 14953 raft_consensus.cc:2229] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:44.070621 14953 raft_consensus.cc:2258] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:44.071154 14953 ts_tablet_manager.cc:1905] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:44.071180 14951 ts_tablet_manager.cc:1905] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:44.071574 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 12760
I20260807 06:33:44.072958 14951 ts_tablet_manager.cc:1918] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:44.073024 14951 log.cc:1196] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/5c210e053daa4cfa93c6a2077faf704d
I20260807 06:33:44.073307 14951 ts_tablet_manager.cc:1939] T 5c210e053daa4cfa93c6a2077faf704d P 9be736c4bcf346bbb551433af3b5cb57: Deleting consensus metadata
I20260807 06:33:44.073292 14953 ts_tablet_manager.cc:1918] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:44.073374 14953 log.cc:1196] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/5c210e053daa4cfa93c6a2077faf704d
I20260807 06:33:44.073688 14953 ts_tablet_manager.cc:1939] T 5c210e053daa4cfa93c6a2077faf704d P 48b2a453116349bca087486129d966f8: Deleting consensus metadata
I20260807 06:33:44.081904 14952 tablet_replica.cc:330] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: stopping tablet replica
I20260807 06:33:44.082116 14952 raft_consensus.cc:2229] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:44.082305 14952 raft_consensus.cc:2258] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:44.085013 14952 ts_tablet_manager.cc:1905] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:44.087256 14952 ts_tablet_manager.cc:1918] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:44.087394 14952 log.cc:1196] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/5c210e053daa4cfa93c6a2077faf704d
I20260807 06:33:44.087759 14952 ts_tablet_manager.cc:1939] T 5c210e053daa4cfa93c6a2077faf704d P 2ce79e330fa4470b8c5d6ac8ff0aff18: Deleting consensus metadata
W20260807 06:33:44.087739 14660 connection.cc:535] server connection from 127.0.0.1:47798 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 06:33:44.087956 14210 connection.cc:535] server connection from 127.0.0.1:58732 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 06:33:44.088310 14433 connection.cc:441] server connection from 127.0.0.1:59394 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59394 (request call id 1) could send its response
I20260807 06:33:44.089798 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:44.070727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.2.243.60:37239)
0807 06:33:44.070842 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:44.070845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:44.081789 (+ 10944us) server_negotiation.cc:407] Connection header received
0807 06:33:44.081829 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:44.081830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:44.081844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:44.081913 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:44.082735 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.083506 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.084697 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.085005 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.085045 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:44.086724 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:44.086726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:44.086726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:44.086734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:44.088430 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:44.088908 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:44.088909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:44.088909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:44.088964 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:44.089168 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:44.089170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:44.089170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:44.089350 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:44.089527 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:44.089612 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 06:33:44.089704 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 06:33:44.097350 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:44.090491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.2.243.61:37469)
0807 06:33:44.090548 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:44.090550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:44.090599 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:33:44.090702 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:44.090704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:44.090717 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:44.090776 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:44.091673 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.092170 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.092970 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.093197 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.093275 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:44.094473 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:44.094475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:44.094475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:44.094483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:44.096066 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:44.096436 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:44.096438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:44.096438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:44.096533 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:44.096744 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:44.096745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:44.096746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:44.096888 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:44.096996 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:44.097148 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 06:33:44.097239 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:33:44.098296 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:44.090241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39768 (local address 127.2.243.60:37239)
0807 06:33:44.090279 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:44.090281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:44.090686 (+ 405us) server_negotiation.cc:407] Connection header received
0807 06:33:44.090779 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:44.090780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:44.090789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:44.090816 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:44.091446 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.092215 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.093331 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:44.093918 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:44.093963 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:44.095340 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:44.095341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:44.095341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:44.095349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:44.097009 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:44.097325 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:44.097326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:44.097326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:44.097430 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:44.097724 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:44.097725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:44.097725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:44.097848 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:44.097920 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:44.098122 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 06:33:44.098204 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:45.067559 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.067285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:37661)
0807 06:33:45.067414 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.067489 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260807 06:33:45.067934 14624 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:42593 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:45.068079 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.067827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:54889)
0807 06:33:45.067941 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.068031 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
W20260807 06:33:45.068253 14398 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:42593 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:45.069044 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.068550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:35723)
0807 06:33:45.068922 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.068996 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":190,"threads_started":1}
W20260807 06:33:45.069273 14860 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:42593 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:45.570711 14971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b9e3c8301267400e901954e239974667)
I20260807 06:33:45.570889 14972 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b9e3c8301267400e901954e239974667)
I20260807 06:33:45.570894 14971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:45.571028 14972 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:45.571208 14971 leader_election.cc:288] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9e3c8301267400e901954e239974667 (127.2.243.62:42593), dc93dcb614524497b8185097e9d8b12c (127.2.243.60:37239)
I20260807 06:33:45.571549 14972 leader_election.cc:288] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9e3c8301267400e901954e239974667 (127.2.243.62:42593), 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61:37469)
I20260807 06:33:45.571774 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.571427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58570)
0807 06:33:45.571664 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.571722 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":79,"threads_started":1}
I20260807 06:33:45.571786 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.571361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58556)
0807 06:33:45.571673 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.571737 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":62,"threads_started":1}
W20260807 06:33:45.571906 14083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
W20260807 06:33:45.572011 13757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:45.572463 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.572246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58574)
0807 06:33:45.572395 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.572427 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 06:33:45.572526 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.572389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58586)
0807 06:33:45.572426 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.572458 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 06:33:45.572613 13757 leader_election.cc:334] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b9e3c8301267400e901954e239974667 (127.2.243.62:42593): Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
W20260807 06:33:45.573069 14083 leader_election.cc:334] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b9e3c8301267400e901954e239974667 (127.2.243.62:42593): Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:45.575664 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.571717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.2.243.60:37239)
0807 06:33:45.571814 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:45.571816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:45.571825 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:45.571854 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:45.571855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:45.571867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:45.571934 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:45.572386 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.573124 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.574533 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.575145 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.575226 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:45.575239 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:45.575477 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 06:33:45.575521 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":80,"thread_start_us":54,"threads_started":1}
I20260807 06:33:45.575672 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.571400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:37239 (local address 127.0.0.1:35692)
0807 06:33:45.571706 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.571717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:45.571779 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:45.572238 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:45.572240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:45.572243 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:45.572302 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:45.572305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.573198 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.573199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:45.574386 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:45.574389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.575256 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.575257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:45.575371 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:45.575377 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:33:45.575461 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:33:45.575560 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":51,"threads_started":1}
I20260807 06:33:45.575793 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.571486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.2.243.61:37469)
0807 06:33:45.571603 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:45.571606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:45.572219 (+ 613us) server_negotiation.cc:407] Connection header received
0807 06:33:45.572250 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:45.572250 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:45.572265 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:45.572347 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:45.572539 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.573367 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.574678 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.575232 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.575294 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:45.575305 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:45.575476 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 06:33:45.575549 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":49,"threads_started":1}
I20260807 06:33:45.575696 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.571440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:37469 (local address 127.0.0.1:35148)
0807 06:33:45.572106 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.572118 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:45.572179 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:45.572362 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:45.572363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:45.572366 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:45.572443 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:45.572444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.573447 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.573448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:45.574622 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:45.574625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:45.575324 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:45.575326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:45.575396 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:45.575401 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:45.575464 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 06:33:45.575511 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":23,"thread_start_us":78,"threads_started":1}
I20260807 06:33:45.576059 14120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1bb181eac61946a58c33dcfdc076a5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dc93dcb614524497b8185097e9d8b12c" is_pre_election: true
I20260807 06:33:45.576174 14120 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb181eac61946a58c33dcfdc076a5b2 in term 1.
I20260807 06:33:45.576414 13762 leader_election.cc:302] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bb181eac61946a58c33dcfdc076a5b2, dc93dcb614524497b8185097e9d8b12c; no voters: b9e3c8301267400e901954e239974667
I20260807 06:33:45.576427 13804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc93dcb614524497b8185097e9d8b12c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb181eac61946a58c33dcfdc076a5b2" is_pre_election: true
I20260807 06:33:45.576566 14971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 06:33:45.576614 13804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc93dcb614524497b8185097e9d8b12c in term 1.
I20260807 06:33:45.576660 14971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b9e3c8301267400e901954e239974667)
I20260807 06:33:45.576705 14971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:33:45.576807 14081 leader_election.cc:302] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bb181eac61946a58c33dcfdc076a5b2, dc93dcb614524497b8185097e9d8b12c; no voters: b9e3c8301267400e901954e239974667
I20260807 06:33:45.576959 14972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 06:33:45.577028 14972 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Starting leader election (detected failure of leader b9e3c8301267400e901954e239974667)
I20260807 06:33:45.577060 14972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:33:45.577741 14971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:45.577903 14971 leader_election.cc:288] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [CANDIDATE]: Term 2 election: Requested vote from peers b9e3c8301267400e901954e239974667 (127.2.243.62:42593), dc93dcb614524497b8185097e9d8b12c (127.2.243.60:37239)
I20260807 06:33:45.577893 14972 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:45.578097 14120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1bb181eac61946a58c33dcfdc076a5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dc93dcb614524497b8185097e9d8b12c"
I20260807 06:33:45.578214 14120 raft_consensus.cc:2379] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1bb181eac61946a58c33dcfdc076a5b2 in current term 2: Already voted for candidate dc93dcb614524497b8185097e9d8b12c in this term.
I20260807 06:33:45.578250 13804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc93dcb614524497b8185097e9d8b12c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb181eac61946a58c33dcfdc076a5b2"
I20260807 06:33:45.578337 13804 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dc93dcb614524497b8185097e9d8b12c in current term 2: Already voted for candidate 1bb181eac61946a58c33dcfdc076a5b2 in this term.
I20260807 06:33:45.578619 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.578161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58594)
0807 06:33:45.578202 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.578592 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 06:33:45.578099 14972 leader_election.cc:288] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 2 election: Requested vote from peers b9e3c8301267400e901954e239974667 (127.2.243.62:42593), 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61:37469)
I20260807 06:33:45.578853 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.578175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58608)
0807 06:33:45.578796 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.578828 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602}
I20260807 06:33:45.579038 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.578912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58618)
0807 06:33:45.578987 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.579021 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 06:33:45.579115 14083 leader_election.cc:334] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b9e3c8301267400e901954e239974667 (127.2.243.62:42593): Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:45.579155 14083 leader_election.cc:302] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc93dcb614524497b8185097e9d8b12c; no voters: 1bb181eac61946a58c33dcfdc076a5b2, b9e3c8301267400e901954e239974667
I20260807 06:33:45.579214 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:45.579115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58626)
0807 06:33:45.579160 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:45.579186 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 06:33:45.579258 14972 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 06:33:45.579285 13757 leader_election.cc:334] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b9e3c8301267400e901954e239974667 (127.2.243.62:42593): Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:45.579353 13757 leader_election.cc:302] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bb181eac61946a58c33dcfdc076a5b2; no voters: b9e3c8301267400e901954e239974667, dc93dcb614524497b8185097e9d8b12c
I20260807 06:33:45.579444 14971 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 06:33:46.068701 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:46.068398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:36175)
0807 06:33:46.068592 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:46.068638 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":135,"threads_started":1}
I20260807 06:33:46.068923 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:46.068624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:47125)
0807 06:33:46.068744 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:46.068868 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260807 06:33:46.069860 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:46.069616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:40077)
0807 06:33:46.069799 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:46.069825 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":52,"threads_started":1}
I20260807 06:33:47.069577 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.069280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:39443)
0807 06:33:47.069432 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.069475 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260807 06:33:47.069598 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.069358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:37823)
0807 06:33:47.069487 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.069538 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":46,"threads_started":1}
I20260807 06:33:47.070544 14996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.070338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:46129)
0807 06:33:47.070458 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.070502 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":53,"threads_started":1}
I20260807 06:33:47.106870 14997 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:47.107019 14997 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:47.107242 14997 leader_election.cc:288] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b9e3c8301267400e901954e239974667 (127.2.243.62:42593), 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61:37469)
I20260807 06:33:47.107551 13804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc93dcb614524497b8185097e9d8b12c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb181eac61946a58c33dcfdc076a5b2" is_pre_election: true
I20260807 06:33:47.107673 13804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc93dcb614524497b8185097e9d8b12c in term 2.
I20260807 06:33:47.107714 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.107392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58826)
0807 06:33:47.107603 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.107643 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":146,"threads_started":1}
I20260807 06:33:47.107887 14081 leader_election.cc:302] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb181eac61946a58c33dcfdc076a5b2, dc93dcb614524497b8185097e9d8b12c; no voters:
I20260807 06:33:47.108003 14997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 06:33:47.108060 14997 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:47.108088 14997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:33:47.108217 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.108079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58840)
0807 06:33:47.108136 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.108186 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 06:33:47.108312 14083 leader_election.cc:334] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b9e3c8301267400e901954e239974667 (127.2.243.62:42593): Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:47.109015 14997 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:47.109174 14997 leader_election.cc:288] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 3 election: Requested vote from peers b9e3c8301267400e901954e239974667 (127.2.243.62:42593), 1bb181eac61946a58c33dcfdc076a5b2 (127.2.243.61:37469)
I20260807 06:33:47.109370 13804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc93dcb614524497b8185097e9d8b12c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb181eac61946a58c33dcfdc076a5b2"
I20260807 06:33:47.109407 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.109299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58850)
0807 06:33:47.109348 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.109375 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 06:33:47.109465 13804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:33:47.109758 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.109668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58856)
0807 06:33:47.109714 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.109738 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:33:47.109843 14083 leader_election.cc:334] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b9e3c8301267400e901954e239974667 (127.2.243.62:42593): Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
I20260807 06:33:47.110212 13804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc93dcb614524497b8185097e9d8b12c in term 3.
I20260807 06:33:47.110381 14081 leader_election.cc:302] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bb181eac61946a58c33dcfdc076a5b2, dc93dcb614524497b8185097e9d8b12c; no voters: b9e3c8301267400e901954e239974667
I20260807 06:33:47.110483 14997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:33:47.110587 14997 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [term 3 LEADER]: Becoming Leader. State: Replica: dc93dcb614524497b8185097e9d8b12c, State: Running, Role: LEADER
I20260807 06:33:47.110667 14997 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } }
I20260807 06:33:47.110994 15005 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc93dcb614524497b8185097e9d8b12c. Latest consensus state: current_term: 3 leader_uuid: "dc93dcb614524497b8185097e9d8b12c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:47.111104 15005 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:47.111315 15008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:47.111833 15008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:47.111927 15008 catalog_manager.cc:1259] Loaded cluster ID: 42e49fac30c54cb99f4487985a373342
I20260807 06:33:47.111963 15008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:47.112172 15008 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:47.112265 15008 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Loaded TSK: 0
I20260807 06:33:47.112432 15008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:33:47.643059 13804 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [term 3 FOLLOWER]: Refusing update from remote peer dc93dcb614524497b8185097e9d8b12c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 13. (index mismatch)
I20260807 06:33:47.643440 15009 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 06:33:47.644090 15015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc93dcb614524497b8185097e9d8b12c. Latest consensus state: current_term: 3 leader_uuid: "dc93dcb614524497b8185097e9d8b12c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:47.644192 15015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:33:47.644850 15017 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "dc93dcb614524497b8185097e9d8b12c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:47.645043 15017 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:47.645108 15015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "dc93dcb614524497b8185097e9d8b12c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "b9e3c8301267400e901954e239974667" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42593 } } peers { permanent_uuid: "1bb181eac61946a58c33dcfdc076a5b2" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 37469 } attrs { promote: false } } peers { permanent_uuid: "dc93dcb614524497b8185097e9d8b12c" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 37239 } attrs { promote: false } } }
I20260807 06:33:47.645202 15015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb181eac61946a58c33dcfdc076a5b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:33:47.645253 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.644958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58866)
0807 06:33:47.645159 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.645208 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260807 06:33:47.645680 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.645595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.0.0.1:58876)
0807 06:33:47.645637 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.645663 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 06:33:47.645792 14083 consensus_peers.cc:487] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c -> Peer b9e3c8301267400e901954e239974667 (127.2.243.62:42593): Couldn't send request to peer b9e3c8301267400e901954e239974667. Status: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 06:33:47.960536 14091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:47.962297 14091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:47.969435 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.965914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38203 (local address 127.0.0.1:40996)
0807 06:33:47.965963 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.965974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:47.966023 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:47.966556 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:47.966557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:47.966560 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:47.966617 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:47.966619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.967634 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.967634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:47.968359 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:47.968361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.969145 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.969146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:47.969237 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:47.969241 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:47.969298 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:33:47.969344 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 06:33:47.969442 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.966264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.2.243.3:38203)
0807 06:33:47.966392 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:47.966394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:47.966403 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:47.966438 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:47.966439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:47.966450 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:47.966682 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:47.966721 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.967551 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.968465 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.969055 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.969129 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:47.969142 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:47.969317 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 06:33:47.969362 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260807 06:33:47.969543 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.966195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.2.243.1:40539)
0807 06:33:47.966346 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:47.966348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:47.966357 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:47.966385 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:47.966386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:47.966399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:47.966621 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:47.966661 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.967181 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.968614 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.969125 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.969395 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:47.969410 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:47.969427 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 06:33:47.969470 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260807 06:33:47.969831 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.965996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40539 (local address 127.0.0.1:42288)
0807 06:33:47.966127 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.966137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:47.966191 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:47.966491 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:47.966492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:47.966495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:47.966554 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:47.966556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.967265 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.967266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:47.968514 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:47.968517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.969232 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.969233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:47.969290 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:47.969293 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:47.969323 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 06:33:47.969353 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":26,"thread_start_us":94,"threads_started":1}
I20260807 06:33:47.969966 14743 tablet_service.cc:1467] Processing CreateTablet for tablet 3890b46b59d4479fbed6fc52dbabe036 (DEFAULT_TABLE table=test-table [id=6d16d17c2af84a1888e979757d5fa8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:47.970120 14743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3890b46b59d4479fbed6fc52dbabe036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:47.970136 14288 tablet_service.cc:1467] Processing CreateTablet for tablet 3890b46b59d4479fbed6fc52dbabe036 (DEFAULT_TABLE table=test-table [id=6d16d17c2af84a1888e979757d5fa8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:47.970203 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.966386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51092 (local address 127.2.243.2:38873)
0807 06:33:47.966502 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:47.966504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:47.966513 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:47.966538 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:47.966539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:47.966549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:47.966612 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:47.966718 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.967440 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.968752 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.969330 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.969396 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:47.969407 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:47.970101 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 06:33:47.970150 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260807 06:33:47.970252 14288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3890b46b59d4479fbed6fc52dbabe036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:47.971776 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.965963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:38873 (local address 127.0.0.1:51092)
0807 06:33:47.966121 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.966134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:47.966211 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:47.966614 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:47.966615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:47.966617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:47.966653 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:47.966654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.967520 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.967521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:47.968615 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:47.968617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:47.969942 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:47.969943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:47.970009 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:47.970013 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:47.971652 (+ 1639us) client_negotiation.cc:240] Negotiation successful
0807 06:33:47.971712 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":116,"threads_started":1}
I20260807 06:33:47.971834 15040 tablet_bootstrap.cc:492] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57: Bootstrap starting.
I20260807 06:33:47.971836 15039 tablet_bootstrap.cc:492] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: Bootstrap starting.
I20260807 06:33:47.972138 14519 tablet_service.cc:1467] Processing CreateTablet for tablet 3890b46b59d4479fbed6fc52dbabe036 (DEFAULT_TABLE table=test-table [id=6d16d17c2af84a1888e979757d5fa8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:47.972297 14519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3890b46b59d4479fbed6fc52dbabe036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:47.972288 15039 tablet_bootstrap.cc:654] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:47.972352 15040 tablet_bootstrap.cc:654] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:47.972950 15040 tablet_bootstrap.cc:492] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57: No bootstrap required, opened a new log
I20260807 06:33:47.972997 15040 ts_tablet_manager.cc:1397] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:47.973117 15040 raft_consensus.cc:348] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:47.973198 15040 raft_consensus.cc:374] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:47.973213 15040 raft_consensus.cc:729] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be736c4bcf346bbb551433af3b5cb57, State: Initialized, Role: FOLLOWER
I20260807 06:33:47.973272 15040 consensus_queue.cc:260] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:47.973402 15040 ts_tablet_manager.cc:1428] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:47.973796 15039 tablet_bootstrap.cc:492] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: No bootstrap required, opened a new log
I20260807 06:33:47.973873 15039 ts_tablet_manager.cc:1397] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:33:47.974117 15039 raft_consensus.cc:348] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:47.974241 15039 raft_consensus.cc:374] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:47.974289 15039 raft_consensus.cc:729] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b2a453116349bca087486129d966f8, State: Initialized, Role: FOLLOWER
I20260807 06:33:47.974311 14400 heartbeater.cc:503] Master 127.2.243.60:37239 was elected leader, sending a full tablet report...
I20260807 06:33:47.974510 15039 consensus_queue.cc:260] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:47.974601 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.973592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:47373)
0807 06:33:47.974360 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.974561 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":110,"threads_started":1}
I20260807 06:33:47.974835 15039 ts_tablet_manager.cc:1428] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 06:33:47.975183 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.974878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:44331)
0807 06:33:47.975090 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.975128 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
I20260807 06:33:47.975337 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.975066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:37579)
0807 06:33:47.975214 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.975292 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260807 06:33:47.975404 14861 heartbeater.cc:503] Master 127.2.243.60:37239 was elected leader, sending a full tablet report...
I20260807 06:33:47.975500 15048 tablet_bootstrap.cc:492] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: Bootstrap starting.
I20260807 06:33:47.975636 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.975549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:40463)
0807 06:33:47.975590 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.975613 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 06:33:47.975940 15048 tablet_bootstrap.cc:654] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:47.976542 15048 tablet_bootstrap.cc:492] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: No bootstrap required, opened a new log
I20260807 06:33:47.976615 15048 ts_tablet_manager.cc:1397] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:47.976749 15048 raft_consensus.cc:348] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:47.976823 15048 raft_consensus.cc:374] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:47.976857 15048 raft_consensus.cc:729] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce79e330fa4470b8c5d6ac8ff0aff18, State: Initialized, Role: FOLLOWER
I20260807 06:33:47.976914 15048 consensus_queue.cc:260] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:47.977097 15048 ts_tablet_manager.cc:1428] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:47.977440 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.977227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:41243)
0807 06:33:47.977361 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.977400 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":92,"threads_started":1}
I20260807 06:33:47.977475 14626 heartbeater.cc:503] Master 127.2.243.60:37239 was elected leader, sending a full tablet report...
I20260807 06:33:47.977929 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:47.977842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:33115)
0807 06:33:47.977880 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:47.977905 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 06:33:48.004365 15057 raft_consensus.cc:482] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:48.004463 15057 raft_consensus.cc:504] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:48.004830 15057 leader_election.cc:288] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b2a453116349bca087486129d966f8 (127.2.243.3:38203), 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1:40539)
I20260807 06:33:48.008832 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.005078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:43577 (local address 127.2.243.3:38203)
0807 06:33:48.005126 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:48.005128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:48.005163 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:33:48.005249 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:48.005250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:48.005263 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:48.005323 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:48.005502 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.006296 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.007869 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.008433 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.008551 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:48.008562 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:48.008721 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 06:33:48.008762 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:33:48.008832 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.005106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:60707 (local address 127.2.243.1:40539)
0807 06:33:48.005171 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:48.005173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:48.005401 (+ 228us) server_negotiation.cc:407] Connection header received
0807 06:33:48.005485 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:48.005486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:48.005500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:48.005553 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:48.005680 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.006359 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.007869 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.008433 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.008543 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:48.008556 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:48.008721 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 06:33:48.008761 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:33:48.008846 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.005017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38203 (local address 127.2.243.2:43577)
0807 06:33:48.005093 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.005106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:48.005182 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:48.005348 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:48.005350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:48.005354 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:48.005427 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:48.005428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.006376 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.006376 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:48.007778 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:48.007782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.008569 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.008570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:48.008634 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:48.008636 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:48.008704 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 06:33:48.008739 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":82}
I20260807 06:33:48.008857 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.005017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40539 (local address 127.2.243.2:60707)
0807 06:33:48.005342 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.005353 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:48.005409 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:48.005570 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:48.005570 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:48.005572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:48.005611 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:48.005612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.006435 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.006436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:48.007778 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:48.007782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.008534 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.008535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:48.008616 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:48.008621 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:48.008683 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:33:48.008737 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":133,"thread_start_us":134,"threads_started":1}
I20260807 06:33:48.009173 14771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3890b46b59d4479fbed6fc52dbabe036" candidate_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b2a453116349bca087486129d966f8" is_pre_election: true
I20260807 06:33:48.009265 14771 raft_consensus.cc:2454] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce79e330fa4470b8c5d6ac8ff0aff18 in term 0.
I20260807 06:33:48.009433 14436 leader_election.cc:302] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce79e330fa4470b8c5d6ac8ff0aff18, 48b2a453116349bca087486129d966f8; no voters:
I20260807 06:33:48.009404 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3890b46b59d4479fbed6fc52dbabe036" candidate_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be736c4bcf346bbb551433af3b5cb57" is_pre_election: true
I20260807 06:33:48.009560 14324 raft_consensus.cc:2454] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce79e330fa4470b8c5d6ac8ff0aff18 in term 0.
I20260807 06:33:48.009574 15057 raft_consensus.cc:2781] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:48.009610 15057 raft_consensus.cc:482] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:48.009644 15057 raft_consensus.cc:3037] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:48.012252 15057 raft_consensus.cc:504] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:48.012426 15057 leader_election.cc:288] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [CANDIDATE]: Term 1 election: Requested vote from peers 48b2a453116349bca087486129d966f8 (127.2.243.3:38203), 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1:40539)
I20260807 06:33:48.012624 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3890b46b59d4479fbed6fc52dbabe036" candidate_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be736c4bcf346bbb551433af3b5cb57"
I20260807 06:33:48.012653 14771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3890b46b59d4479fbed6fc52dbabe036" candidate_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b2a453116349bca087486129d966f8"
I20260807 06:33:48.012741 14771 raft_consensus.cc:3037] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:48.012744 14324 raft_consensus.cc:3037] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:48.013309 14771 raft_consensus.cc:2454] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce79e330fa4470b8c5d6ac8ff0aff18 in term 1.
I20260807 06:33:48.013477 14436 leader_election.cc:302] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce79e330fa4470b8c5d6ac8ff0aff18, 48b2a453116349bca087486129d966f8; no voters:
I20260807 06:33:48.013598 15057 raft_consensus.cc:2781] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:48.013622 14324 raft_consensus.cc:2454] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce79e330fa4470b8c5d6ac8ff0aff18 in term 1.
I20260807 06:33:48.013793 15057 raft_consensus.cc:686] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 LEADER]: Becoming Leader. State: Replica: 2ce79e330fa4470b8c5d6ac8ff0aff18, State: Running, Role: LEADER
I20260807 06:33:48.013880 15057 consensus_queue.cc:237] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } }
I20260807 06:33:48.014117 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.014006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:39001)
0807 06:33:48.014045 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.014084 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 06:33:48.014590 14091 catalog_manager.cc:5495] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2). New cstate: current_term: 1 leader_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce79e330fa4470b8c5d6ac8ff0aff18" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260807 06:33:48.028261 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.025969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.2.243.2:38873)
0807 06:33:48.026024 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:48.026026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:48.026053 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:33:48.026083 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:48.026084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:48.026103 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:48.026158 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:48.026298 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.026815 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.027593 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:48.027771 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:48.027844 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:48.027959 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:48.028062 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:48.028113 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:48.028158 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:33:48.028174 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:33:48.029460 14771 raft_consensus.cc:1261] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Refusing update from remote peer 2ce79e330fa4470b8c5d6ac8ff0aff18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:48.029489 14324 raft_consensus.cc:1261] T 3890b46b59d4479fbed6fc52dbabe036 P 9be736c4bcf346bbb551433af3b5cb57 [term 1 FOLLOWER]: Refusing update from remote peer 2ce79e330fa4470b8c5d6ac8ff0aff18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:48.029815 15057 consensus_queue.cc:1035] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b2a453116349bca087486129d966f8" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:48.029980 15064 consensus_queue.cc:1035] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9be736c4bcf346bbb551433af3b5cb57" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:48.030951 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.030827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:44045)
0807 06:33:48.030884 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.030915 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 06:33:48.031360 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.031260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:50127)
0807 06:33:48.031301 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.031330 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:33:48.031879 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.031739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.3:58803)
0807 06:33:48.031823 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.031849 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 06:33:48.032171 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.032060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:43369)
0807 06:33:48.032101 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.032134 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 06:33:48.034142 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.033835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.1:60721)
0807 06:33:48.034080 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.034112 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226}
I20260807 06:33:48.034518 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.033525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42593 (local address 127.2.243.2:48167)
0807 06:33:48.034461 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.034492 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":920}
I20260807 06:33:48.034894 14103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:48.034999 14103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:48.036358 14103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc93dcb614524497b8185097e9d8b12c: Sending DeleteTablet for 3 replicas of tablet 3890b46b59d4479fbed6fc52dbabe036
I20260807 06:33:48.036682 14288 tablet_service.cc:1514] Processing DeleteTablet for tablet 3890b46b59d4479fbed6fc52dbabe036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:48 UTC) from {username='oryx', principal='oryx/127.2.243.60@KRBTEST.COM'} at 127.0.0.1:42288
I20260807 06:33:48.036701 14519 tablet_service.cc:1514] Processing DeleteTablet for tablet 3890b46b59d4479fbed6fc52dbabe036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:48 UTC) from {username='oryx', principal='oryx/127.2.243.60@KRBTEST.COM'} at 127.0.0.1:51092
I20260807 06:33:48.036697 14743 tablet_service.cc:1514] Processing DeleteTablet for tablet 3890b46b59d4479fbed6fc52dbabe036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:48 UTC) from {username='oryx', principal='oryx/127.2.243.60@KRBTEST.COM'} at 127.0.0.1:40996
I20260807 06:33:48.037158 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 14049
I20260807 06:33:48.040624 15086 tablet_replica.cc:330] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: stopping tablet replica
I20260807 06:33:48.040696 15086 raft_consensus.cc:2229] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:48.040745 15086 raft_consensus.cc:2258] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:48.040877 15083 tablet_replica.cc:330] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: stopping tablet replica
I20260807 06:33:48.040932 15083 raft_consensus.cc:2229] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:48.041030 15083 raft_consensus.cc:2258] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:48.049356 15086 ts_tablet_manager.cc:1905] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:48.049383 15083 ts_tablet_manager.cc:1905] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:48.051326 15083 ts_tablet_manager.cc:1918] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:48.051322 15086 ts_tablet_manager.cc:1918] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:48.051395 15086 log.cc:1196] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/3890b46b59d4479fbed6fc52dbabe036
I20260807 06:33:48.051505 15083 log.cc:1196] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/3890b46b59d4479fbed6fc52dbabe036
I20260807 06:33:48.051676 15086 ts_tablet_manager.cc:1939] T 3890b46b59d4479fbed6fc52dbabe036 P 48b2a453116349bca087486129d966f8: Deleting consensus metadata
I20260807 06:33:48.051831 15083 ts_tablet_manager.cc:1939] T 3890b46b59d4479fbed6fc52dbabe036 P 2ce79e330fa4470b8c5d6ac8ff0aff18: Deleting consensus metadata
I20260807 06:33:48.052110 14087 catalog_manager.cc:4841] TS 48b2a453116349bca087486129d966f8 (127.2.243.3:38203): tablet 3890b46b59d4479fbed6fc52dbabe036 (table test-table [id=6d16d17c2af84a1888e979757d5fa8f9]) successfully deleted
I20260807 06:33:48.052176 14083 catalog_manager.cc:4841] TS 2ce79e330fa4470b8c5d6ac8ff0aff18 (127.2.243.2:38873): tablet 3890b46b59d4479fbed6fc52dbabe036 (table test-table [id=6d16d17c2af84a1888e979757d5fa8f9]) successfully deleted
I20260807 06:33:48.056355 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:48.056221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40539 (local address 127.0.0.1:42300)
0807 06:33:48.056266 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:48.056292 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 06:33:48.056439 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 14410
W20260807 06:33:48.061218 14081 catalog_manager.cc:4584] TS 9be736c4bcf346bbb551433af3b5cb57 (127.2.243.1:40539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3890b46b59d4479fbed6fc52dbabe036: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40539: connect: Connection refused (error 111)
I20260807 06:33:48.069568 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 14634
I20260807 06:33:48.080137 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 13558
I20260807 06:33:48.091720 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 13871
2026-08-07T06:33:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7483 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:48 dist-test-slave-rwrg krb5kdc[15089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:48 dist-test-slave-rwrg krb5kdc[15089](info): set up 2 sockets
Aug 07 06:33:48 dist-test-slave-rwrg krb5kdc[15089](info): commencing operation
krb5kdc: starting...
W20260807 06:33:49.355782 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:49 dist-test-slave-rwrg krb5kdc[15089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab.
Entry for principal oryx/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab.
I20260807 06:33:49.481189 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39229
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab
--principal=oryx/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:49.562906 15111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:49.563139 15111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:49.563189 15111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:49.564314 15111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:49.564380 15111 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:33:49.564401 15111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:49.564422 15111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:49.564455 15111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:49.564477 15111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:49.565972 15111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39229
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.62.keytab
--principal=oryx/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:49.566260 15111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:49.566479 15111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:49.571434 15126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:49.571440 15131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:49.571441 15124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:49.571741 15111 server_base.cc:1034] running on GCE node
I20260807 06:33:49.571975 15111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:49.572216 15111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:49.573383 15111 hybrid_clock.cc:648] HybridClock initialized: now 1786084429573367 us; error 31 us; skew 500 ppm
Aug 07 06:33:49 dist-test-slave-rwrg krb5kdc[15089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:49.576081 15111 init.cc:377] Logged in from keytab as oryx/127.2.243.62@KRBTEST.COM (short username oryx)
I20260807 06:33:49.576691 15111 webserver.cc:460] Webserver started at http://127.2.243.62:35933/ using document root <none> and password file <none>
I20260807 06:33:49.576941 15111 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:49.577004 15111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:49.577162 15111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:49.578271 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "3e337680be7b4f3c906f57147e4a530f"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "098ac50f552abb727b12209e850d1a78"
server_key_iv: "bc0e76ee0ecce761df59e038e8007fc1"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.578668 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "3e337680be7b4f3c906f57147e4a530f"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "098ac50f552abb727b12209e850d1a78"
server_key_iv: "bc0e76ee0ecce761df59e038e8007fc1"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.580132 15111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:49.581025 15141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:49.581254 15111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:33:49.581377 15111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "3e337680be7b4f3c906f57147e4a530f"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "098ac50f552abb727b12209e850d1a78"
server_key_iv: "bc0e76ee0ecce761df59e038e8007fc1"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.581455 15111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:49.599283 15111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:49.601217 15111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:49.601398 15111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:49.606670 15111 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39229
I20260807 06:33:49.606729 15245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39229 every 8 connection(s)
I20260807 06:33:49.607151 15111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:49.607841 15246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:49.610185 15246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Bootstrap starting.
I20260807 06:33:49.610737 15246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:49.610977 15246 log.cc:824] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:49.611696 15246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: No bootstrap required, opened a new log
I20260807 06:33:49.613801 15246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e337680be7b4f3c906f57147e4a530f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39229 } }
I20260807 06:33:49.613936 15246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:49.613960 15246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e337680be7b4f3c906f57147e4a530f, State: Initialized, Role: FOLLOWER
I20260807 06:33:49.614111 15246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e337680be7b4f3c906f57147e4a530f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39229 } }
I20260807 06:33:49.614200 15246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:49.614223 15246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:49.614250 15246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:49.614773 15246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e337680be7b4f3c906f57147e4a530f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39229 } }
I20260807 06:33:49.614862 15246 leader_election.cc:302] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e337680be7b4f3c906f57147e4a530f; no voters:
I20260807 06:33:49.614989 15246 leader_election.cc:288] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:49.615085 15252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:49.615252 15252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [term 1 LEADER]: Becoming Leader. State: Replica: 3e337680be7b4f3c906f57147e4a530f, State: Running, Role: LEADER
I20260807 06:33:49.615268 15246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:49.615388 15252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e337680be7b4f3c906f57147e4a530f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39229 } }
I20260807 06:33:49.615816 15253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e337680be7b4f3c906f57147e4a530f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e337680be7b4f3c906f57147e4a530f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39229 } } }
I20260807 06:33:49.615844 15254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e337680be7b4f3c906f57147e4a530f. Latest consensus state: current_term: 1 leader_uuid: "3e337680be7b4f3c906f57147e4a530f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e337680be7b4f3c906f57147e4a530f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39229 } } }
I20260807 06:33:49.616037 15254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:49.616256 15271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:49.616647 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 15111
I20260807 06:33:49.616742 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:49.616868 15271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:49.616988 3020 external_mini_cluster.cc:1428] Setting key 23a0ef257f00915851380ab4af273052
I20260807 06:33:49.617660 15253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:49.618530 15271 catalog_manager.cc:1347] Generated new cluster ID: 4dd1d5cc820f4307af9aadaf3f7923f2
I20260807 06:33:49.618582 15271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:33:49 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:49.624986 15271 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:49.625701 15271 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:49.627281 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:49.617485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.2.243.62:39229)
0807 06:33:49.617843 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:49.617851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:49.617876 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:33:49.617996 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:49.618011 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:49.618078 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:49.618230 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:49.618955 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.620014 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.620877 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.621319 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.621439 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:49.623811 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:49.623815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:49.623816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:49.623834 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:49.625717 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:49.626104 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:49.626108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:49.626108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:49.626209 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:49.626437 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:49.626438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:49.626438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:49.626667 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:49.626834 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:49.626907 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 06:33:49.627072 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":76,"threads_started":1}
I20260807 06:33:49.643253 15271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Generated new TSK 0
I20260807 06:33:49.643491 15271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab.
Entry for principal oryx/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab.
I20260807 06:33:49.676258 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39229
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab
--principal=oryx/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:49.791651 15295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:49.791899 15295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:49.791926 15295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:49.793151 15295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:49.793192 15295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:49.793213 15295 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:49.793231 15295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:49.794756 15295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab
--principal=oryx/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:39229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:49.795001 15295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:49.795248 15295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:49.800181 15310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:49.800181 15301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:49.800244 15304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:49.800434 15295 server_base.cc:1034] running on GCE node
I20260807 06:33:49.800724 15295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:49.800963 15295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:49.802132 15295 hybrid_clock.cc:648] HybridClock initialized: now 1786084429802117 us; error 28 us; skew 500 ppm
Aug 07 06:33:49 dist-test-slave-rwrg krb5kdc[15089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:49.805208 15295 init.cc:377] Logged in from keytab as oryx/127.2.243.1@KRBTEST.COM (short username oryx)
I20260807 06:33:49.805646 15295 webserver.cc:460] Webserver started at http://127.2.243.1:45313/ using document root <none> and password file <none>
I20260807 06:33:49.805840 15295 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:49.805884 15295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:49.805982 15295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:49.807056 15295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "8537f495c3d84146b627fcbc8d7ace01"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "0cbd6a48dfa7a9e88679c9a9b5874fc0"
server_key_iv: "340b4638585a668cbf0446c67743449f"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.807385 15295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "8537f495c3d84146b627fcbc8d7ace01"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "0cbd6a48dfa7a9e88679c9a9b5874fc0"
server_key_iv: "340b4638585a668cbf0446c67743449f"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.808868 15295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:49.809690 15322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:49.809875 15295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:49.809988 15295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "8537f495c3d84146b627fcbc8d7ace01"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "0cbd6a48dfa7a9e88679c9a9b5874fc0"
server_key_iv: "340b4638585a668cbf0446c67743449f"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.810079 15295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:49.836737 15295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:49.838799 15295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:49.838966 15295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:49.839488 15295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:49.839564 15295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:49.839614 15295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:49.839640 15295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:33:49 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:49.847050 15295 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:39103
I20260807 06:33:49.847406 15295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:49.847548 15530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:39103 every 8 connection(s)
I20260807 06:33:49.850135 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:49.840099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:33393 (local address 127.2.243.62:39229)
0807 06:33:49.840179 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:49.840183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:49.840340 (+ 157us) server_negotiation.cc:407] Connection header received
0807 06:33:49.840887 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:49.840888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:49.840901 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:49.840982 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:49.841651 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.842488 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.844286 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.844653 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.844734 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:49.847474 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:49.847475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:49.847476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:49.847484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:49.848902 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:49.849302 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:49.849303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:49.849303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:49.849354 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:49.849618 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:49.849618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:49.849619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:49.849718 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:49.849767 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:49.850009 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 06:33:49.850072 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:33:49.850308 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:49.840006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39229 (local address 127.2.243.1:33393)
0807 06:33:49.840182 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:49.840222 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:49.840785 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:49.841070 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:49.841073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:49.841365 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:49.841551 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:49.841556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.842620 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.842621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:49.844166 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:49.844170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.844771 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.844772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:49.844934 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:49.845426 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:49.845438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:49.847369 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:49.848969 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:49.848976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:49.848978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:49.849192 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:49.849412 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:49.849413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:49.849413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:49.849559 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:49.849774 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:49.849779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:49.849890 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 06:33:49.849970 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:33:49.850088 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260807 06:33:49.850945 15532 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39229
I20260807 06:33:49.851049 15532 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:49.851230 15532 heartbeater.cc:511] Master 127.2.243.62:39229 requested a full tablet report, sending...
I20260807 06:33:49.851766 15167 ts_manager.cc:194] Registered new tserver with Master: 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103)
I20260807 06:33:49.852643 15167 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:33393
I20260807 06:33:49.853209 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 15295
I20260807 06:33:49.853310 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:33:49.853528 3020 external_mini_cluster.cc:1428] Setting key 26974062f58d83c2ac53e3839fad65ea
I20260807 06:33:49.858305 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:49.852111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:39097 (local address 127.2.243.62:39229)
0807 06:33:49.852147 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:49.852149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:49.852339 (+ 190us) server_negotiation.cc:407] Connection header received
0807 06:33:49.852365 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:49.852366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:49.852376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:49.852440 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:49.852795 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.853477 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.854365 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.854705 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.854783 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:49.855689 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:49.855690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:49.855690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:49.855697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:49.857131 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:49.857479 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:49.857480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:49.857481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:49.857543 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:49.857776 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:49.857777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:49.857777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:49.857887 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:49.857959 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:49.858138 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 06:33:49.858214 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:49.858320 15543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:49.852053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39229 (local address 127.2.243.1:39097)
0807 06:33:49.852261 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:49.852271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:49.852332 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:49.852442 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:49.852443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:49.852649 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:49.852705 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:49.852707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.853561 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.853562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:49.854288 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:49.854291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:49.854785 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:49.854786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:49.854833 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:49.855290 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:49.855297 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:49.855583 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:49.857234 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:49.857235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:49.857235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:49.857404 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:49.857630 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:49.857631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:49.857632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:49.857688 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:49.857965 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:49.857965 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:49.858033 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 06:33:49.858112 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:33:49.858216 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab.
Entry for principal oryx/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab.
I20260807 06:33:49.880359 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39229
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab
--principal=oryx/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:49.962600 15548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:49.962831 15548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:49.962872 15548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:49.964040 15548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:49.964112 15548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:49.964150 15548 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:49.964205 15548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:49.965788 15548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab
--principal=oryx/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:39229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:49.966068 15548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:49.966297 15548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:49.971443 15557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:49.971438 15560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:49.971449 15558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:49.971509 15548 server_base.cc:1034] running on GCE node
I20260807 06:33:49.971879 15548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:49.972167 15548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:49.973348 15548 hybrid_clock.cc:648] HybridClock initialized: now 1786084429973326 us; error 39 us; skew 500 ppm
Aug 07 06:33:49 dist-test-slave-rwrg krb5kdc[15089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:49.976166 15548 init.cc:377] Logged in from keytab as oryx/127.2.243.2@KRBTEST.COM (short username oryx)
I20260807 06:33:49.976637 15548 webserver.cc:460] Webserver started at http://127.2.243.2:42923/ using document root <none> and password file <none>
I20260807 06:33:49.976892 15548 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:49.976961 15548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:49.977116 15548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:49.978183 15548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "215478c5795744ecbc1229dbda325baf"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "9a485ac0acdfee95c6e8bad2e9a6ad6e"
server_key_iv: "3a3085e621a3ade85b30233fab9b5402"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.978554 15548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "215478c5795744ecbc1229dbda325baf"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "9a485ac0acdfee95c6e8bad2e9a6ad6e"
server_key_iv: "3a3085e621a3ade85b30233fab9b5402"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.979909 15548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:49.980736 15577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:49.980945 15548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:49.981058 15548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "215478c5795744ecbc1229dbda325baf"
format_stamp: "Formatted at 2026-08-07 06:33:49 on dist-test-slave-rwrg"
server_key: "9a485ac0acdfee95c6e8bad2e9a6ad6e"
server_key_iv: "3a3085e621a3ade85b30233fab9b5402"
server_key_version: "encryptionkey@0"
I20260807 06:33:49.981156 15548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:49.996711 15548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:49.998809 15548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:49.998929 15548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:49.999402 15548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:49.999469 15548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:49.999505 15548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:49.999541 15548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:50.007525 15548 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:36605
I20260807 06:33:50.007596 15771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:36605 every 8 connection(s)
I20260807 06:33:50.008050 15548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:50.009977 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.000062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:44601 (local address 127.2.243.62:39229)
0807 06:33:50.000146 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.000149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.000309 (+ 160us) server_negotiation.cc:407] Connection header received
0807 06:33:50.000869 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.000870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.000886 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.000976 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.001588 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.002427 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.003491 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.003819 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.003890 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.006909 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:50.006911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:50.006911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:50.006920 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:50.008752 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.009114 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.009115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.009116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.009174 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.009366 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.009366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.009367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.009526 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:50.009584 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.009821 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.009909 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 06:33:50.010087 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:49.999987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39229 (local address 127.2.243.2:44601)
0807 06:33:50.000163 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.000200 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.000767 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.001070 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.001074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.001297 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:50.001472 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.001476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.002555 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.002556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.003391 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.003394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.003895 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.003896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.004046 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.004712 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:50.004728 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:50.006812 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:50.008826 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.008830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.008832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.009044 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.009242 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.009242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.009242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.009329 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.009587 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:50.009591 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:50.009701 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.009783 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.009901 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260807 06:33:50.010574 15773 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39229
I20260807 06:33:50.010659 15773 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:50.010835 15773 heartbeater.cc:511] Master 127.2.243.62:39229 requested a full tablet report, sending...
I20260807 06:33:50.011243 15166 ts_manager.cc:194] Registered new tserver with Master: 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605)
I20260807 06:33:50.011826 15166 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:44601
I20260807 06:33:50.015758 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 15548
I20260807 06:33:50.015887 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:50.016134 3020 external_mini_cluster.cc:1428] Setting key b06270ea86f5c4bfecc290f8c38c8744
I20260807 06:33:50.017354 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.011604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39229 (local address 127.2.243.2:38463)
0807 06:33:50.011740 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.011750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.011808 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.011956 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.011956 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.012150 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:50.012223 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.012225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.012973 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.012974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.013479 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.013480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.013889 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.013890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.013974 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.014385 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:50.014389 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:50.014697 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:50.016240 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.016240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.016241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.016433 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.016706 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.016707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.016707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.016773 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.017097 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:50.017098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:50.017138 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.017205 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.017283 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260807 06:33:50.017469 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.011684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:38463 (local address 127.2.243.62:39229)
0807 06:33:50.011721 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.011722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.011799 (+ 77us) server_negotiation.cc:407] Connection header received
0807 06:33:50.011869 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.011870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.011877 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.011948 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.012325 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.012895 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.013590 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.013804 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.013887 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.014790 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:50.014791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:50.014791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:50.014798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:50.016131 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.016578 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.016580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.016580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.016635 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.016844 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.016845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.016845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.017032 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:50.017108 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.017255 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.017334 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab.
Entry for principal oryx/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab.
I20260807 06:33:50.043783 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39229
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab
--principal=oryx/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:50.127465 15787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:50.127717 15787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:50.127741 15787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:50.128903 15787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:50.128943 15787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:50.128963 15787 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:33:50.128980 15787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:50.130447 15787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab
--principal=oryx/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.243.62:39229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:50.130630 15787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:50.130798 15787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:50.135802 15796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:50.135799 15800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:50.135875 15798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:50.136130 15787 server_base.cc:1034] running on GCE node
I20260807 06:33:50.136374 15787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:50.136662 15787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:50.137843 15787 hybrid_clock.cc:648] HybridClock initialized: now 1786084430137824 us; error 31 us; skew 500 ppm
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084430, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:50.140523 15787 init.cc:377] Logged in from keytab as oryx/127.2.243.3@KRBTEST.COM (short username oryx)
I20260807 06:33:50.140973 15787 webserver.cc:460] Webserver started at http://127.2.243.3:34451/ using document root <none> and password file <none>
I20260807 06:33:50.141249 15787 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:50.141319 15787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:50.141466 15787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:50.142526 15787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "8ccbba0a2664499886b78b63b40cffc4"
format_stamp: "Formatted at 2026-08-07 06:33:50 on dist-test-slave-rwrg"
server_key: "465837958cd385aaaf93c0977db6c175"
server_key_iv: "350d198da8ab8667d7010c96652faca7"
server_key_version: "encryptionkey@0"
I20260807 06:33:50.142885 15787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "8ccbba0a2664499886b78b63b40cffc4"
format_stamp: "Formatted at 2026-08-07 06:33:50 on dist-test-slave-rwrg"
server_key: "465837958cd385aaaf93c0977db6c175"
server_key_iv: "350d198da8ab8667d7010c96652faca7"
server_key_version: "encryptionkey@0"
I20260807 06:33:50.144412 15787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:50.145177 15811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.145404 15787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:50.145551 15787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "8ccbba0a2664499886b78b63b40cffc4"
format_stamp: "Formatted at 2026-08-07 06:33:50 on dist-test-slave-rwrg"
server_key: "465837958cd385aaaf93c0977db6c175"
server_key_iv: "350d198da8ab8667d7010c96652faca7"
server_key_version: "encryptionkey@0"
I20260807 06:33:50.145677 15787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:50.157401 15787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:50.159521 15787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:50.159801 15787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:50.160430 15787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:50.160552 15787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.160648 15787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:50.160692 15787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084430, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:50.169171 15787 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:42873
I20260807 06:33:50.169322 16019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:42873 every 8 connection(s)
I20260807 06:33:50.169845 15787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:50.172086 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.160950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:39381 (local address 127.2.243.62:39229)
0807 06:33:50.161016 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.161018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.161234 (+ 216us) server_negotiation.cc:407] Connection header received
0807 06:33:50.161800 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.161801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.161813 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.161872 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.162624 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.163399 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.164912 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.165223 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.165294 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.168823 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:50.168824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:50.168825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:50.168833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:50.170711 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.171126 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.171128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.171128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.171186 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.171445 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.171446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.171447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.171571 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:50.171647 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.171916 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.172001 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:33:50.172266 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.160939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39229 (local address 127.2.243.3:39381)
0807 06:33:50.161115 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.161149 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.161691 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.161961 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.161964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.162214 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:50.162535 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.162543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.163516 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.163517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.164820 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.164823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.165331 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.165332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.165511 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.166080 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:50.166092 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:50.168724 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:50.170808 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.170815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.170817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.171043 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.171270 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.171271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.171271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.171380 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.171638 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:50.171644 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:50.171780 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.171880 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.172041 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260807 06:33:50.172770 16020 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39229
I20260807 06:33:50.172871 16020 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:50.173050 16020 heartbeater.cc:511] Master 127.2.243.62:39229 requested a full tablet report, sending...
I20260807 06:33:50.173504 15167 ts_manager.cc:194] Registered new tserver with Master: 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873)
I20260807 06:33:50.174007 15167 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:39381
I20260807 06:33:50.178833 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.173793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:55327 (local address 127.2.243.62:39229)
0807 06:33:50.173833 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.173835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.173908 (+ 73us) server_negotiation.cc:407] Connection header received
0807 06:33:50.173961 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.173962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.173970 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.174011 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.174362 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.174828 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.175543 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.175737 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.175811 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.176647 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:50.176649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:50.176649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:50.176656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:50.177710 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.178063 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.178064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.178064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.178124 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.178334 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.178334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.178335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.178440 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:50.178523 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.178670 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.178742 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:33:50.178848 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.173698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39229 (local address 127.2.243.3:55327)
0807 06:33:50.173838 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.173849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.173908 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.174032 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.174033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.174226 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:50.174280 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.174282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.174922 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.174923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.175435 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.175437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.175811 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.175812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.175884 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.176240 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:50.176243 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:50.176564 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:50.177797 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.177798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.177799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.177969 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.178193 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.178193 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.178194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.178251 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.178507 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:50.178507 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:50.178552 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.178643 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.178744 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260807 06:33:50.179477 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 15787
I20260807 06:33:50.179557 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:50.179788 3020 external_mini_cluster.cc:1428] Setting key 6c721dbfa6f9af8085b9eabd579ceb5f
I20260807 06:33:50.180770 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084430, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:33:50.202181 15289 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:50.196873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.2.243.62:39229)
0807 06:33:50.196940 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.196942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.196968 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:33:50.197078 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.197079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.197088 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.197143 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.197749 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.198236 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.198973 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.199162 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.199239 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.202075 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43328: BlockingRecv error: recv got EOF from 127.0.0.1:43328 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084430, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:50.209270 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.202921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43344 (local address 127.2.243.62:39229)
0807 06:33:50.202987 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.202989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.203045 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:33:50.203085 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.203085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.203093 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.203141 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.203602 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.204040 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.204752 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.204915 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.204964 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.206880 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:50.206881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:50.206881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:50.206887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:50.208310 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.208664 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.208664 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.208665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.208714 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.208866 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.208867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.208867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.208954 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:50.209020 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.209113 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.209168 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:33:50.210834 15167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:50.212654 15167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:50.219594 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.215078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39103 (local address 127.0.0.1:34254)
0807 06:33:50.215371 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.215384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.215435 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.215791 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.215794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.215796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:50.215858 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.215860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.217152 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.217153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.218295 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.218297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.219271 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.219272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.219363 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.219378 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.219448 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.219502 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260807 06:33:50.219806 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.215245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:36605 (local address 127.0.0.1:44384)
0807 06:33:50.216172 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.216183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.216264 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.216436 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.216437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.216439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:50.216476 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.216478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.217594 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.217596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.218848 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.218850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.219546 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.219547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.219614 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.219620 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.219671 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.219725 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"thread_start_us":79,"threads_started":1}
I20260807 06:33:50.219810 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.215042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:42873 (local address 127.0.0.1:40648)
0807 06:33:50.215206 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.215247 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.215314 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.216086 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.216087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.216088 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:50.216129 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.216130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.217421 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.217422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.218589 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.218592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.219584 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.219585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.219648 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.219652 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.219689 (+ 37us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.219731 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":48,"thread_start_us":60,"threads_started":1}
I20260807 06:33:50.219936 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.215365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.2.243.1:39103)
0807 06:33:50.215585 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.215588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.215597 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:50.215669 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.215672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.215689 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.215775 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:50.215930 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.217059 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.218383 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.219087 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.219159 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.219232 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.219775 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.219817 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260807 06:33:50.220216 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.215139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.2.243.3:42873)
0807 06:33:50.215900 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.215905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.215915 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:50.215951 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.215955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.215970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.216087 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:50.216211 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.217326 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.218677 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.219481 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.219961 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.220046 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.220073 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.220114 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":89,"threads_started":1}
I20260807 06:33:50.220288 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.215363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.2.243.2:36605)
0807 06:33:50.215541 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.215546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.216254 (+ 708us) server_negotiation.cc:407] Connection header received
0807 06:33:50.216330 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.216333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.216348 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.216428 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:50.216679 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.217513 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.218966 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.219452 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.219543 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.219614 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.220018 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.220070 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260807 06:33:50.220717 15420 tablet_service.cc:1467] Processing CreateTablet for tablet cbc537f1b3074053a3048ae2703a65d7 (DEFAULT_TABLE table=test-table [id=c9a6d20aaf2e47b1998f08e22478ad2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.221144 15420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbc537f1b3074053a3048ae2703a65d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.221021 15663 tablet_service.cc:1467] Processing CreateTablet for tablet cbc537f1b3074053a3048ae2703a65d7 (DEFAULT_TABLE table=test-table [id=c9a6d20aaf2e47b1998f08e22478ad2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.221369 15663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbc537f1b3074053a3048ae2703a65d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.224850 15898 tablet_service.cc:1467] Processing CreateTablet for tablet cbc537f1b3074053a3048ae2703a65d7 (DEFAULT_TABLE table=test-table [id=c9a6d20aaf2e47b1998f08e22478ad2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.225205 15898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbc537f1b3074053a3048ae2703a65d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.225256 16067 tablet_bootstrap.cc:492] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Bootstrap starting.
I20260807 06:33:50.225780 16067 tablet_bootstrap.cc:654] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.226001 16067 log.cc:824] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:50.226316 16066 tablet_bootstrap.cc:492] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Bootstrap starting.
I20260807 06:33:50.226765 16067 tablet_bootstrap.cc:492] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: No bootstrap required, opened a new log
I20260807 06:33:50.226824 16067 ts_tablet_manager.cc:1397] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:50.226936 16066 tablet_bootstrap.cc:654] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.227250 16066 log.cc:824] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:50.228087 16066 tablet_bootstrap.cc:492] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: No bootstrap required, opened a new log
I20260807 06:33:50.228108 16071 tablet_bootstrap.cc:492] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Bootstrap starting.
I20260807 06:33:50.228143 16066 ts_tablet_manager.cc:1397] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:50.228689 16071 tablet_bootstrap.cc:654] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.228972 16071 log.cc:824] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:50.228863 16067 raft_consensus.cc:348] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.229033 16067 raft_consensus.cc:374] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.229110 16067 raft_consensus.cc:729] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8537f495c3d84146b627fcbc8d7ace01, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.229249 16067 consensus_queue.cc:260] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.229558 16067 ts_tablet_manager.cc:1428] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:33:50.229629 15532 heartbeater.cc:503] Master 127.2.243.62:39229 was elected leader, sending a full tablet report...
I20260807 06:33:50.230157 16066 raft_consensus.cc:348] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.230300 16066 raft_consensus.cc:374] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.230345 16066 raft_consensus.cc:729] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215478c5795744ecbc1229dbda325baf, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.230501 16066 consensus_queue.cc:260] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.230798 16066 ts_tablet_manager.cc:1428] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:33:50.230821 15773 heartbeater.cc:503] Master 127.2.243.62:39229 was elected leader, sending a full tablet report...
I20260807 06:33:50.231449 16071 tablet_bootstrap.cc:492] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: No bootstrap required, opened a new log
I20260807 06:33:50.231530 16071 ts_tablet_manager.cc:1397] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:33:50.233028 16071 raft_consensus.cc:348] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.233152 16071 raft_consensus.cc:374] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.233174 16071 raft_consensus.cc:729] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ccbba0a2664499886b78b63b40cffc4, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.233300 16071 consensus_queue.cc:260] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.233566 16071 ts_tablet_manager.cc:1428] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:33:50.233654 16020 heartbeater.cc:503] Master 127.2.243.62:39229 was elected leader, sending a full tablet report...
W20260807 06:33:50.258682 15774 tablet.cc:2369] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:50.305649 16074 raft_consensus.cc:482] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:50.305778 16074 raft_consensus.cc:504] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.306093 16074 leader_election.cc:288] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873), 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605)
I20260807 06:33:50.309172 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.306423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:45369 (local address 127.2.243.3:42873)
0807 06:33:50.306486 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.306488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.306499 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:50.306630 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.306632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.306645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.306693 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:50.306890 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.307355 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.308302 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.308799 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.308884 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.308895 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.309063 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.309097 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:33:50.309206 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.306363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:42873 (local address 127.2.243.1:45369)
0807 06:33:50.306429 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.306444 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.306507 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.306743 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.306744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.306745 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:50.306818 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.306819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.307443 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.307444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.308198 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.308200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.308884 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.308885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.308974 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.308982 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.309058 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.309097 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:33:50.309199 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.306627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:43769 (local address 127.2.243.2:36605)
0807 06:33:50.306670 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.306672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.306786 (+ 114us) server_negotiation.cc:407] Connection header received
0807 06:33:50.306854 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.306855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.306867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.306939 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:50.307106 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.307545 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.308431 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.308859 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.308934 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.308944 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.309097 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.309132 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:33:50.309239 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.306472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:36605 (local address 127.2.243.1:43769)
0807 06:33:50.306700 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.306710 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.306795 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.306945 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.306946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.306947 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:50.307035 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.307037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.307628 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.307629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.308339 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.308341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.308950 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.308951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.309026 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.309029 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.309091 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.309126 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":166,"threads_started":1}
I20260807 06:33:50.309616 15689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbc537f1b3074053a3048ae2703a65d7" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf" is_pre_election: true
I20260807 06:33:50.309779 15689 raft_consensus.cc:2454] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 0.
I20260807 06:33:50.309774 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbc537f1b3074053a3048ae2703a65d7" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ccbba0a2664499886b78b63b40cffc4" is_pre_election: true
I20260807 06:33:50.309927 15932 raft_consensus.cc:2454] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 0.
I20260807 06:33:50.310019 15325 leader_election.cc:302] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215478c5795744ecbc1229dbda325baf, 8537f495c3d84146b627fcbc8d7ace01; no voters:
I20260807 06:33:50.310144 16074 raft_consensus.cc:2781] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:50.310201 16074 raft_consensus.cc:482] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:50.310227 16074 raft_consensus.cc:3037] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.311033 16074 raft_consensus.cc:504] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.311169 16074 leader_election.cc:288] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 election: Requested vote from peers 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873), 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605)
I20260807 06:33:50.311362 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbc537f1b3074053a3048ae2703a65d7" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ccbba0a2664499886b78b63b40cffc4"
I20260807 06:33:50.311465 15932 raft_consensus.cc:3037] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.312252 15932 raft_consensus.cc:2454] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 1.
I20260807 06:33:50.312376 15689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbc537f1b3074053a3048ae2703a65d7" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf"
I20260807 06:33:50.312453 15325 leader_election.cc:302] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8537f495c3d84146b627fcbc8d7ace01, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:50.312465 15689 raft_consensus.cc:3037] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.312615 16074 raft_consensus.cc:2781] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:50.312769 16074 raft_consensus.cc:686] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 LEADER]: Becoming Leader. State: Replica: 8537f495c3d84146b627fcbc8d7ace01, State: Running, Role: LEADER
I20260807 06:33:50.312865 16074 consensus_queue.cc:237] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } }
I20260807 06:33:50.313362 15689 raft_consensus.cc:2454] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 1.
I20260807 06:33:50.313795 15166 catalog_manager.cc:5495] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1). New cstate: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } health_report { overall_health: HEALTHY } } }
I20260807 06:33:50.334065 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.331621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.2.243.1:39103)
0807 06:33:50.331694 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.331696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.331703 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:33:50.331734 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.331734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.331751 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.331809 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:50.331942 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.332453 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.333323 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.333506 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.333583 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.333726 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.333823 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.333882 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.333920 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.333938 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:33:50.336362 15689 raft_consensus.cc:1261] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Refusing update from remote peer 8537f495c3d84146b627fcbc8d7ace01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:50.336362 15932 raft_consensus.cc:1261] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Refusing update from remote peer 8537f495c3d84146b627fcbc8d7ace01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:50.336751 16074 consensus_queue.cc:1035] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:50.336933 16080 consensus_queue.cc:1035] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:50.340138 16086 mvcc.cc:205] Tried to move back new op lower bound from 7315801826655076352 to 7315801826562306048. Current Snapshot: MvccSnapshot[applied={T|T < 7315801826655076352}]
I20260807 06:33:50.342007 15166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:50.342095 15166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:50.342875 15166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Sending DeleteTablet for 3 replicas of tablet cbc537f1b3074053a3048ae2703a65d7
I20260807 06:33:50.343199 15898 tablet_service.cc:1514] Processing DeleteTablet for tablet cbc537f1b3074053a3048ae2703a65d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:50 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:40648
I20260807 06:33:50.343235 15420 tablet_service.cc:1514] Processing DeleteTablet for tablet cbc537f1b3074053a3048ae2703a65d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:50 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:34254
I20260807 06:33:50.343384 15166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:50.343520 15166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:50.343559 15663 tablet_service.cc:1514] Processing DeleteTablet for tablet cbc537f1b3074053a3048ae2703a65d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:50 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:44384
I20260807 06:33:50.343771 16102 tablet_replica.cc:330] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: stopping tablet replica
I20260807 06:33:50.343866 16102 raft_consensus.cc:2229] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:50.344009 16102 raft_consensus.cc:2258] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:50.344038 16101 tablet_replica.cc:330] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: stopping tablet replica
I20260807 06:33:50.344226 16101 raft_consensus.cc:2229] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:50.344398 16101 raft_consensus.cc:2258] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:50.344516 16102 ts_tablet_manager.cc:1905] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:50.344673 16100 tablet_replica.cc:330] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: stopping tablet replica
I20260807 06:33:50.344815 16100 raft_consensus.cc:2229] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:50.345054 16100 raft_consensus.cc:2258] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:50.345296 15420 tablet_service.cc:1467] Processing CreateTablet for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 (DEFAULT_TABLE table=test-table [id=541444e74587403d9cec0ec8199d4f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.345371 15663 tablet_service.cc:1467] Processing CreateTablet for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 (DEFAULT_TABLE table=test-table [id=541444e74587403d9cec0ec8199d4f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.345425 15420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.345611 15663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.345654 16101 ts_tablet_manager.cc:1905] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:50.346390 16102 ts_tablet_manager.cc:1918] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:50.346454 16102 log.cc:1196] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/cbc537f1b3074053a3048ae2703a65d7
I20260807 06:33:50.346714 16102 ts_tablet_manager.cc:1939] T cbc537f1b3074053a3048ae2703a65d7 P 215478c5795744ecbc1229dbda325baf: Deleting consensus metadata
I20260807 06:33:50.347062 15898 tablet_service.cc:1467] Processing CreateTablet for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 (DEFAULT_TABLE table=test-table [id=541444e74587403d9cec0ec8199d4f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.347162 15898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.347833 16101 ts_tablet_manager.cc:1918] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:50.347891 16101 log.cc:1196] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/cbc537f1b3074053a3048ae2703a65d7
I20260807 06:33:50.348362 16071 tablet_bootstrap.cc:492] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Bootstrap starting.
I20260807 06:33:50.348461 16067 tablet_bootstrap.cc:492] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Bootstrap starting.
I20260807 06:33:50.348482 16101 ts_tablet_manager.cc:1939] T cbc537f1b3074053a3048ae2703a65d7 P 8ccbba0a2664499886b78b63b40cffc4: Deleting consensus metadata
I20260807 06:33:50.348803 16071 tablet_bootstrap.cc:654] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.348974 16067 tablet_bootstrap.cc:654] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.349363 16066 tablet_bootstrap.cc:492] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: Bootstrap starting.
I20260807 06:33:50.349474 16071 tablet_bootstrap.cc:492] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: No bootstrap required, opened a new log
I20260807 06:33:50.349519 16071 ts_tablet_manager.cc:1397] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:50.349671 16066 tablet_bootstrap.cc:654] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.349653 16071 raft_consensus.cc:348] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.349715 16071 raft_consensus.cc:374] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.349737 16071 raft_consensus.cc:729] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ccbba0a2664499886b78b63b40cffc4, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.349784 16071 consensus_queue.cc:260] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.349867 16071 ts_tablet_manager.cc:1428] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:50.350353 15148 catalog_manager.cc:4841] TS 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873): tablet cbc537f1b3074053a3048ae2703a65d7 (table test-table [id=c9a6d20aaf2e47b1998f08e22478ad2f]) successfully deleted
I20260807 06:33:50.350572 15148 catalog_manager.cc:4841] TS 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605): tablet cbc537f1b3074053a3048ae2703a65d7 (table test-table [id=c9a6d20aaf2e47b1998f08e22478ad2f]) successfully deleted
I20260807 06:33:50.350901 16067 tablet_bootstrap.cc:492] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: No bootstrap required, opened a new log
I20260807 06:33:50.350901 16066 tablet_bootstrap.cc:492] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: No bootstrap required, opened a new log
I20260807 06:33:50.350947 16067 ts_tablet_manager.cc:1397] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:33:50.350947 16066 ts_tablet_manager.cc:1397] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:33:50.351085 16067 raft_consensus.cc:348] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.351101 16066 raft_consensus.cc:348] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.351146 16067 raft_consensus.cc:374] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.351156 16066 raft_consensus.cc:374] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.351179 16066 raft_consensus.cc:729] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215478c5795744ecbc1229dbda325baf, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.351169 16067 raft_consensus.cc:729] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8537f495c3d84146b627fcbc8d7ace01, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.351217 16066 consensus_queue.cc:260] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.351235 16067 consensus_queue.cc:260] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.351305 16066 ts_tablet_manager.cc:1428] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.351336 16067 ts_tablet_manager.cc:1428] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.351794 16100 ts_tablet_manager.cc:1905] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:50.353165 16100 ts_tablet_manager.cc:1918] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:50.353259 16100 log.cc:1196] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/cbc537f1b3074053a3048ae2703a65d7
I20260807 06:33:50.353582 16100 ts_tablet_manager.cc:1939] T cbc537f1b3074053a3048ae2703a65d7 P 8537f495c3d84146b627fcbc8d7ace01: Deleting consensus metadata
I20260807 06:33:50.354035 15146 catalog_manager.cc:4841] TS 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103): tablet cbc537f1b3074053a3048ae2703a65d7 (table test-table [id=c9a6d20aaf2e47b1998f08e22478ad2f]) successfully deleted
W20260807 06:33:50.420471 16022 tablet.cc:2369] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:33:50.598445 15534 tablet.cc:2369] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:33:50.651698 16077 raft_consensus.cc:482] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:50.651805 16077 raft_consensus.cc:504] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.652230 16077 leader_election.cc:288] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103), 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605)
I20260807 06:33:50.656262 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.652627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:42667 (local address 127.2.243.2:36605)
0807 06:33:50.652685 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.652687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.652762 (+ 75us) server_negotiation.cc:407] Connection header received
0807 06:33:50.652792 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.652793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.652802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.652856 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:50.653079 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.653832 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.655324 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.655854 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.655932 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.655943 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.656154 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.656193 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:33:50.656262 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.652617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:39971 (local address 127.2.243.1:39103)
0807 06:33:50.652685 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.652687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.652698 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:50.652757 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.652758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.652770 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.652843 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:50.653086 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.653860 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.655323 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.655867 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.655932 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.655943 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.656157 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.656196 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:33:50.656311 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.652433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39103 (local address 127.2.243.3:39971)
0807 06:33:50.652520 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.652534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.652623 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.652881 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.652882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.652884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:50.653000 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.653003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.653979 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.653980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.655208 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.655211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.655959 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.655960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.656063 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.656068 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.656137 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.656193 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":35}
I20260807 06:33:50.656328 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.652433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:36605 (local address 127.2.243.3:42667)
0807 06:33:50.652684 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.652695 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.652750 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.652881 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.652882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.652884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:50.653001 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.653003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.653971 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.653972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.655208 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.655211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.655979 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.655980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.656063 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.656069 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.656137 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.656193 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":113,"thread_start_us":103,"threads_started":1}
I20260807 06:33:50.656687 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d8d7957fb8346f3b68f293dc0ebe0c8" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8537f495c3d84146b627fcbc8d7ace01" is_pre_election: true
I20260807 06:33:50.656829 15448 raft_consensus.cc:2454] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 0.
I20260807 06:33:50.656787 15689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d8d7957fb8346f3b68f293dc0ebe0c8" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf" is_pre_election: true
I20260807 06:33:50.656951 15689 raft_consensus.cc:2454] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 0.
I20260807 06:33:50.657059 15815 leader_election.cc:302] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8537f495c3d84146b627fcbc8d7ace01, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:50.657220 16077 raft_consensus.cc:2781] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:50.657294 16077 raft_consensus.cc:482] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:50.657326 16077 raft_consensus.cc:3037] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.658028 16077 raft_consensus.cc:504] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.658200 16077 leader_election.cc:288] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 election: Requested vote from peers 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103), 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605)
I20260807 06:33:50.658425 15689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d8d7957fb8346f3b68f293dc0ebe0c8" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf"
I20260807 06:33:50.658416 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d8d7957fb8346f3b68f293dc0ebe0c8" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8537f495c3d84146b627fcbc8d7ace01"
I20260807 06:33:50.658494 15448 raft_consensus.cc:3037] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.658515 15689 raft_consensus.cc:3037] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.659157 15689 raft_consensus.cc:2454] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 1.
I20260807 06:33:50.659173 15448 raft_consensus.cc:2454] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 1.
I20260807 06:33:50.659351 15815 leader_election.cc:302] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8537f495c3d84146b627fcbc8d7ace01, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:50.659442 16077 raft_consensus.cc:2781] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:50.659547 16077 raft_consensus.cc:686] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 LEADER]: Becoming Leader. State: Replica: 8ccbba0a2664499886b78b63b40cffc4, State: Running, Role: LEADER
I20260807 06:33:50.659647 16077 consensus_queue.cc:237] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.660422 15166 catalog_manager.cc:5495] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3). New cstate: current_term: 1 leader_uuid: "8ccbba0a2664499886b78b63b40cffc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } health_report { overall_health: HEALTHY } } }
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084429, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.62@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:50.725907 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.713314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.2.243.62:39229)
0807 06:33:50.713457 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.713460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.713573 (+ 113us) server_negotiation.cc:407] Connection header received
0807 06:33:50.713662 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.713662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.713677 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.713754 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.714311 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.714849 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.715525 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.715727 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.715817 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.723245 (+ 7428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:50.723246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:50.723246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:50.723253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:50.724783 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.725176 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.725177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.725177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.725244 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.725448 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.725449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.725449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.725575 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:50.725633 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.725756 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.725835 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260807 06:33:50.726048 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.713247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39229 (local address 127.0.0.1:43356)
0807 06:33:50.713466 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.713478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.713599 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.713771 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.713772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.714176 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:50.714221 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.714224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.714921 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.714922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.715421 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.715423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.715838 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.715838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.715921 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.721664 (+ 5743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:50.721669 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:50.723149 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:50.724858 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.724865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.724867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.725084 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.725324 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:50.725324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:50.725325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:50.725383 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:50.725636 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:50.725642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:50.725686 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.725740 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.725910 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":71,"thread_start_us":110,"threads_started":1}
I20260807 06:33:50.728300 15166 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:43356:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:33:50.728565 15166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:50.730213 15663 tablet_service.cc:1467] Processing CreateTablet for tablet 51324912dd8b4e6988da2cb0d8a6d52a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0d190d5a55442e6aa6fe511931b7de7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:50.730214 15898 tablet_service.cc:1467] Processing CreateTablet for tablet 51324912dd8b4e6988da2cb0d8a6d52a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0d190d5a55442e6aa6fe511931b7de7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:50.730201 15420 tablet_service.cc:1467] Processing CreateTablet for tablet 51324912dd8b4e6988da2cb0d8a6d52a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0d190d5a55442e6aa6fe511931b7de7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:33:50.730386 15420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51324912dd8b4e6988da2cb0d8a6d52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.730386 15898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51324912dd8b4e6988da2cb0d8a6d52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.730388 15663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51324912dd8b4e6988da2cb0d8a6d52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.731628 16066 tablet_bootstrap.cc:492] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: Bootstrap starting.
I20260807 06:33:50.731921 16067 tablet_bootstrap.cc:492] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: Bootstrap starting.
I20260807 06:33:50.731941 16071 tablet_bootstrap.cc:492] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: Bootstrap starting.
I20260807 06:33:50.732106 16066 tablet_bootstrap.cc:654] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.732262 16067 tablet_bootstrap.cc:654] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.732391 16071 tablet_bootstrap.cc:654] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.733014 16066 tablet_bootstrap.cc:492] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: No bootstrap required, opened a new log
I20260807 06:33:50.733067 16066 ts_tablet_manager.cc:1397] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:50.733078 16071 tablet_bootstrap.cc:492] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: No bootstrap required, opened a new log
I20260807 06:33:50.733115 16071 ts_tablet_manager.cc:1397] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:50.733129 16067 tablet_bootstrap.cc:492] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: No bootstrap required, opened a new log
I20260807 06:33:50.733177 16067 ts_tablet_manager.cc:1397] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:33:50.733253 16071 raft_consensus.cc:348] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.733235 16066 raft_consensus.cc:348] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.733309 16066 raft_consensus.cc:374] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.733307 16071 raft_consensus.cc:374] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.733330 16071 raft_consensus.cc:729] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ccbba0a2664499886b78b63b40cffc4, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.733330 16066 raft_consensus.cc:729] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215478c5795744ecbc1229dbda325baf, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.733296 16067 raft_consensus.cc:348] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.733376 16067 raft_consensus.cc:374] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.733404 16067 raft_consensus.cc:729] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8537f495c3d84146b627fcbc8d7ace01, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.733381 16071 consensus_queue.cc:260] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.733381 16066 consensus_queue.cc:260] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.733448 16067 consensus_queue.cc:260] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.733471 16071 ts_tablet_manager.cc:1428] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.733470 16066 ts_tablet_manager.cc:1428] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.733534 16067 ts_tablet_manager.cc:1428] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:50.733664 16077 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.733753 16077 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:50.733704 16076 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.733811 16076 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:50.733852 16074 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.733917 16074 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:50.767392 16074 raft_consensus.cc:482] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:50.767541 16074 raft_consensus.cc:504] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.767840 16074 leader_election.cc:288] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605), 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873)
I20260807 06:33:50.768029 15689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51324912dd8b4e6988da2cb0d8a6d52a" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf" is_pre_election: true
I20260807 06:33:50.768090 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51324912dd8b4e6988da2cb0d8a6d52a" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ccbba0a2664499886b78b63b40cffc4" is_pre_election: true
I20260807 06:33:50.768152 15689 raft_consensus.cc:2454] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 0.
I20260807 06:33:50.768182 15932 raft_consensus.cc:2454] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 0.
I20260807 06:33:50.768324 15325 leader_election.cc:302] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8537f495c3d84146b627fcbc8d7ace01, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:50.768424 16074 raft_consensus.cc:2781] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:50.768532 16074 raft_consensus.cc:482] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:50.768560 16074 raft_consensus.cc:3037] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.769243 16074 raft_consensus.cc:504] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.769399 16074 leader_election.cc:288] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 election: Requested vote from peers 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605), 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873)
I20260807 06:33:50.769583 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51324912dd8b4e6988da2cb0d8a6d52a" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ccbba0a2664499886b78b63b40cffc4"
I20260807 06:33:50.769621 15689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51324912dd8b4e6988da2cb0d8a6d52a" candidate_uuid: "8537f495c3d84146b627fcbc8d7ace01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf"
I20260807 06:33:50.769665 15932 raft_consensus.cc:3037] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.769707 15689 raft_consensus.cc:3037] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:50.770282 15932 raft_consensus.cc:2454] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 1.
I20260807 06:33:50.770356 15689 raft_consensus.cc:2454] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8537f495c3d84146b627fcbc8d7ace01 in term 1.
I20260807 06:33:50.770483 15325 leader_election.cc:302] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8537f495c3d84146b627fcbc8d7ace01, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:50.770586 16074 raft_consensus.cc:2781] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:50.770648 16074 raft_consensus.cc:686] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [term 1 LEADER]: Becoming Leader. State: Replica: 8537f495c3d84146b627fcbc8d7ace01, State: Running, Role: LEADER
I20260807 06:33:50.770711 16074 consensus_queue.cc:237] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.771018 16092 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: TxnStatusTablet state changed. Reason: New leader 8537f495c3d84146b627fcbc8d7ace01. Latest consensus state: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.771121 16092 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:50.771276 16137 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:33:50.771338 15166 catalog_manager.cc:5495] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1). New cstate: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } health_report { overall_health: UNKNOWN } } }
I20260807 06:33:50.771421 16137 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:33:50.783149 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.779773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.2.243.1:39103)
0807 06:33:50.779934 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.779936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.779945 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:33:50.779989 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.779990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.780004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.780077 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:50.780272 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.781040 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.782286 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.782598 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.782665 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.782871 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.782907 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.782961 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.783047 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.783070 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260807 06:33:50.783211 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.779714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39103 (local address 127.0.0.1:34262)
0807 06:33:50.779779 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.779792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.779837 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.780115 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.780116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.780118 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:50.780178 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.780181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.781221 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.781222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.782165 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.782169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.782685 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.782686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.782752 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.782775 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.782978 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.782990 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.783046 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.783109 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:33:50.784302 15689 raft_consensus.cc:1261] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Refusing update from remote peer 8537f495c3d84146b627fcbc8d7ace01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:50.784308 15932 raft_consensus.cc:1261] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Refusing update from remote peer 8537f495c3d84146b627fcbc8d7ace01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:50.784560 16092 consensus_queue.cc:1035] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:50.784672 16074 consensus_queue.cc:1035] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:50.785152 16076 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: TxnStatusTablet state changed. Reason: New leader 8537f495c3d84146b627fcbc8d7ace01. Latest consensus state: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.785249 16076 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:50.785290 16077 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: TxnStatusTablet state changed. Reason: New leader 8537f495c3d84146b627fcbc8d7ace01. Latest consensus state: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.785379 16077 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:50.785964 16080 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.786046 16080 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:50.786111 16084 mvcc.cc:205] Tried to move back new op lower bound from 7315801828490727424 to 7315801828437192704. Current Snapshot: MvccSnapshot[applied={T|T < 7315801828490727424}]
I20260807 06:33:50.786098 16088 mvcc.cc:205] Tried to move back new op lower bound from 7315801828490727424 to 7315801828437192704. Current Snapshot: MvccSnapshot[applied={T|T < 7315801828490727424}]
I20260807 06:33:50.786206 16077 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.786269 16077 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 8ccbba0a2664499886b78b63b40cffc4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:50.794450 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.790986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.2.243.3:42873)
0807 06:33:50.791187 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.791189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.791195 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:33:50.791235 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.791236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.791251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.791328 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:50.791476 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.792240 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.793353 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.793669 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.793745 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.793888 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.794016 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.794083 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.794321 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.794341 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260807 06:33:50.797192 16092 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.797317 16092 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 8537f495c3d84146b627fcbc8d7ace01: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:33:50.797989 16076 tablet_replica.cc:439] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8537f495c3d84146b627fcbc8d7ace01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } } }
I20260807 06:33:50.798097 16076 tablet_replica.cc:442] T 51324912dd8b4e6988da2cb0d8a6d52a P 215478c5795744ecbc1229dbda325baf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:33:50.801028 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.797483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:46699 (local address 127.2.243.1:39103)
0807 06:33:50.797604 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.797606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.797616 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:50.797638 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.797639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.797650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.797710 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:50.798046 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.798823 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.800047 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.800392 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.800474 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.800753 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.800787 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.800846 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.800920 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.800955 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 06:33:50.801056 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.797326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39103 (local address 127.2.243.3:46699)
0807 06:33:50.797479 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.797490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.797541 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.797857 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.797858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.797861 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:50.797942 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.797944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.798968 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.798969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.799963 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.799967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.800529 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.800530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.800602 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.800622 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.800860 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.800863 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.800917 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.800968 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260807 06:33:50.804221 15689 raft_consensus.cc:1261] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Refusing update from remote peer 8ccbba0a2664499886b78b63b40cffc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:50.804523 16077 consensus_queue.cc:1035] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:50.805807 15448 raft_consensus.cc:1261] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Refusing update from remote peer 8ccbba0a2664499886b78b63b40cffc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:50.806089 16077 consensus_queue.cc:1035] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:50.818994 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.813433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:60445 (local address 127.2.243.3:42873)
0807 06:33:50.813472 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.813475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.813583 (+ 108us) server_negotiation.cc:407] Connection header received
0807 06:33:50.813605 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.813606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.813618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.813662 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:50.814345 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.815100 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.818132 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.818425 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.818465 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.818734 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.818768 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.818795 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.818897 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.818918 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:33:50.819793 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.813313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:42873 (local address 127.2.243.1:60445)
0807 06:33:50.813467 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:50.813479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:50.813544 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:50.814197 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:50.814198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:50.814199 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:33:50.814262 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.814263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.815213 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.815214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.818028 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:50.818031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.818520 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.818521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:50.818599 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.818615 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:33:50.818834 (+ 219us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:33:50.818836 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:33:50.819122 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 06:33:50.819701 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260807 06:33:50.829430 15166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:50.829538 15166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:50.830173 15166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Sending DeleteTablet for 3 replicas of tablet 8d8d7957fb8346f3b68f293dc0ebe0c8
I20260807 06:33:50.830443 15898 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:50 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:40648
I20260807 06:33:50.830556 15420 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:50 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:34254
I20260807 06:33:50.830559 16101 tablet_replica.cc:330] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: stopping tablet replica
I20260807 06:33:50.830629 16101 raft_consensus.cc:2229] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:50.830657 16100 tablet_replica.cc:330] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: stopping tablet replica
I20260807 06:33:50.830713 16101 raft_consensus.cc:2258] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:50.830719 16100 raft_consensus.cc:2229] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:50.830757 16100 raft_consensus.cc:2258] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:50.831035 16101 ts_tablet_manager.cc:1905] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:50.831108 16100 ts_tablet_manager.cc:1905] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:50.832583 15663 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:50 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:44384
I20260807 06:33:50.832861 16102 tablet_replica.cc:330] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: stopping tablet replica
I20260807 06:33:50.832934 16102 raft_consensus.cc:2229] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:50.833047 16100 ts_tablet_manager.cc:1918] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:50.833096 16102 raft_consensus.cc:2258] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:50.833106 16100 log.cc:1196] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/8d8d7957fb8346f3b68f293dc0ebe0c8
I20260807 06:33:50.833343 16100 ts_tablet_manager.cc:1939] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8537f495c3d84146b627fcbc8d7ace01: Deleting consensus metadata
I20260807 06:33:50.833772 15146 catalog_manager.cc:4841] TS 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103): tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 (table test-table [id=541444e74587403d9cec0ec8199d4f41]) successfully deleted
I20260807 06:33:50.834146 16101 ts_tablet_manager.cc:1918] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:50.834203 16101 log.cc:1196] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/8d8d7957fb8346f3b68f293dc0ebe0c8
I20260807 06:33:50.834538 16101 ts_tablet_manager.cc:1939] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 8ccbba0a2664499886b78b63b40cffc4: Deleting consensus metadata
I20260807 06:33:50.834937 15148 catalog_manager.cc:4841] TS 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873): tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 (table test-table [id=541444e74587403d9cec0ec8199d4f41]) successfully deleted
I20260807 06:33:50.835148 16102 ts_tablet_manager.cc:1905] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:50.836799 16102 ts_tablet_manager.cc:1918] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:50.836869 16102 log.cc:1196] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/8d8d7957fb8346f3b68f293dc0ebe0c8
I20260807 06:33:50.837102 16102 ts_tablet_manager.cc:1939] T 8d8d7957fb8346f3b68f293dc0ebe0c8 P 215478c5795744ecbc1229dbda325baf: Deleting consensus metadata
I20260807 06:33:50.837450 15148 catalog_manager.cc:4841] TS 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605): tablet 8d8d7957fb8346f3b68f293dc0ebe0c8 (table test-table [id=541444e74587403d9cec0ec8199d4f41]) successfully deleted
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:33:50.874341 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:50.867007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43362 (local address 127.2.243.62:39229)
0807 06:33:50.867064 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.867067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.867238 (+ 171us) server_negotiation.cc:407] Connection header received
0807 06:33:50.867285 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.867286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.867303 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.867372 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.868111 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.868983 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.870159 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.870447 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.870526 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.874274 (+ 3748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43362: BlockingRecv error: recv got EOF from 127.0.0.1:43362 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 06:33:50 dist-test-slave-rwrg krb5kdc[15089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM
I20260807 06:33:50.884698 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:50.875264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.2.243.62:39229)
0807 06:33:50.875301 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:50.875303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:50.875407 (+ 104us) server_negotiation.cc:407] Connection header received
0807 06:33:50.875529 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:50.875530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:50.875546 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:50.875614 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:50.876207 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.876978 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.877956 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:50.878268 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:50.878342 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:50.881407 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:50.881409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:50.881409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:50.881418 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:50.883190 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.883692 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.883694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.883694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.883760 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:50.883998 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:50.883999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:50.883999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:50.884140 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:50.884246 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:50.884428 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 06:33:50.884533 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:50.886620 15166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:33:50.886793 15166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:50.888206 15420 tablet_service.cc:1467] Processing CreateTablet for tablet a2a07edd2e424d3bb6a2bded97991de6 (DEFAULT_TABLE table=test-table [id=e9ed2d16518b47569bf109a3b2494d65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.888283 15898 tablet_service.cc:1467] Processing CreateTablet for tablet a2a07edd2e424d3bb6a2bded97991de6 (DEFAULT_TABLE table=test-table [id=e9ed2d16518b47569bf109a3b2494d65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.888356 15420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2a07edd2e424d3bb6a2bded97991de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.888417 15898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2a07edd2e424d3bb6a2bded97991de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.888718 15663 tablet_service.cc:1467] Processing CreateTablet for tablet a2a07edd2e424d3bb6a2bded97991de6 (DEFAULT_TABLE table=test-table [id=e9ed2d16518b47569bf109a3b2494d65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:50.888828 15663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2a07edd2e424d3bb6a2bded97991de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:50.889909 16067 tablet_bootstrap.cc:492] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: Bootstrap starting.
I20260807 06:33:50.890146 16066 tablet_bootstrap.cc:492] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: Bootstrap starting.
I20260807 06:33:50.890219 16067 tablet_bootstrap.cc:654] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.890540 16066 tablet_bootstrap.cc:654] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.892480 16071 tablet_bootstrap.cc:492] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: Bootstrap starting.
I20260807 06:33:50.892973 16071 tablet_bootstrap.cc:654] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:50.893666 16066 tablet_bootstrap.cc:492] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: No bootstrap required, opened a new log
I20260807 06:33:50.893715 16066 ts_tablet_manager.cc:1397] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:33:50.893872 16066 raft_consensus.cc:348] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.893936 16066 raft_consensus.cc:374] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.893957 16066 raft_consensus.cc:729] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215478c5795744ecbc1229dbda325baf, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.894001 16066 consensus_queue.cc:260] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.894095 16066 ts_tablet_manager.cc:1428] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.894591 16071 tablet_bootstrap.cc:492] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: No bootstrap required, opened a new log
I20260807 06:33:50.894639 16071 ts_tablet_manager.cc:1397] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:33:50.894776 16071 raft_consensus.cc:348] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.894832 16071 raft_consensus.cc:374] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.894852 16071 raft_consensus.cc:729] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ccbba0a2664499886b78b63b40cffc4, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.894888 16071 consensus_queue.cc:260] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.894971 16071 ts_tablet_manager.cc:1428] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:50.896297 16067 tablet_bootstrap.cc:492] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: No bootstrap required, opened a new log
I20260807 06:33:50.896366 16067 ts_tablet_manager.cc:1397] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260807 06:33:50.896472 16067 raft_consensus.cc:348] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.896608 16067 raft_consensus.cc:374] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:50.896638 16067 raft_consensus.cc:729] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8537f495c3d84146b627fcbc8d7ace01, State: Initialized, Role: FOLLOWER
I20260807 06:33:50.896682 16067 consensus_queue.cc:260] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:50.896786 16067 ts_tablet_manager.cc:1428] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:51.070739 16076 raft_consensus.cc:482] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:51.070871 16076 raft_consensus.cc:504] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.071261 16076 leader_election.cc:288] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103), 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873)
I20260807 06:33:51.074918 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:51.071479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:42873 (local address 127.2.243.2:47679)
0807 06:33:51.071654 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:51.071665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:51.071737 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:51.071896 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:51.071897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:51.071900 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:51.071968 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:51.071970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.072906 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.072907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:51.074050 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:51.074052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.074690 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.074691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:51.074733 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:51.074736 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:33:51.074775 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 06:33:51.074804 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":33,"thread_start_us":80,"threads_started":1}
I20260807 06:33:51.074954 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:51.071567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:47679 (local address 127.2.243.3:42873)
0807 06:33:51.071632 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:51.071634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:51.071753 (+ 119us) server_negotiation.cc:407] Connection header received
0807 06:33:51.071796 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:51.071797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:51.071809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:51.071886 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:51.072048 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.072788 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.074232 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.074596 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.074632 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:51.074641 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:51.074819 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 06:33:51.074870 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:33:51.075062 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:51.071479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39103 (local address 127.2.243.2:39127)
0807 06:33:51.071675 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:51.071694 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:51.071811 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:51.071989 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:51.071990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:51.071992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:33:51.072114 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:51.072116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.072952 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.072953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:51.074016 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:51.074019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.074738 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.074739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:51.074834 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:51.074838 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:33:51.074903 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:33:51.074944 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":5,"thread_start_us":76,"threads_started":1}
I20260807 06:33:51.075069 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:51.071567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:39127 (local address 127.2.243.1:39103)
0807 06:33:51.071632 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:51.071635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:51.071821 (+ 186us) server_negotiation.cc:407] Connection header received
0807 06:33:51.071892 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:51.071893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:51.071902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:51.071962 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:33:51.072225 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.072851 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.074184 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.074622 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.074668 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:51.074677 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:51.074926 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 06:33:51.074974 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:33:51.075210 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a07edd2e424d3bb6a2bded97991de6" candidate_uuid: "215478c5795744ecbc1229dbda325baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ccbba0a2664499886b78b63b40cffc4" is_pre_election: true
I20260807 06:33:51.075322 15932 raft_consensus.cc:2454] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 215478c5795744ecbc1229dbda325baf in term 0.
I20260807 06:33:51.075513 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a07edd2e424d3bb6a2bded97991de6" candidate_uuid: "215478c5795744ecbc1229dbda325baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8537f495c3d84146b627fcbc8d7ace01" is_pre_election: true
I20260807 06:33:51.075541 15581 leader_election.cc:302] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215478c5795744ecbc1229dbda325baf, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:51.075618 15448 raft_consensus.cc:2454] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 215478c5795744ecbc1229dbda325baf in term 0.
I20260807 06:33:51.075685 16076 raft_consensus.cc:2781] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:51.075721 16076 raft_consensus.cc:482] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:51.075748 16076 raft_consensus.cc:3037] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:51.076615 16076 raft_consensus.cc:504] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.076759 16076 leader_election.cc:288] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [CANDIDATE]: Term 1 election: Requested vote from peers 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103), 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873)
I20260807 06:33:51.076928 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a07edd2e424d3bb6a2bded97991de6" candidate_uuid: "215478c5795744ecbc1229dbda325baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ccbba0a2664499886b78b63b40cffc4"
I20260807 06:33:51.076943 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2a07edd2e424d3bb6a2bded97991de6" candidate_uuid: "215478c5795744ecbc1229dbda325baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8537f495c3d84146b627fcbc8d7ace01"
I20260807 06:33:51.077034 15448 raft_consensus.cc:3037] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:51.077047 15932 raft_consensus.cc:3037] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:51.077705 15448 raft_consensus.cc:2454] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 215478c5795744ecbc1229dbda325baf in term 1.
I20260807 06:33:51.077714 15932 raft_consensus.cc:2454] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 215478c5795744ecbc1229dbda325baf in term 1.
I20260807 06:33:51.077899 15579 leader_election.cc:302] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215478c5795744ecbc1229dbda325baf, 8537f495c3d84146b627fcbc8d7ace01; no voters:
I20260807 06:33:51.078049 16076 raft_consensus.cc:2781] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:51.078231 16076 raft_consensus.cc:686] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 1 LEADER]: Becoming Leader. State: Replica: 215478c5795744ecbc1229dbda325baf, State: Running, Role: LEADER
I20260807 06:33:51.078322 16076 consensus_queue.cc:237] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.079107 15163 catalog_manager.cc:5495] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf reported cstate change: term changed from 0 to 1, leader changed from <none> to 215478c5795744ecbc1229dbda325baf (127.2.243.2). New cstate: current_term: 1 leader_uuid: "215478c5795744ecbc1229dbda325baf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } health_report { overall_health: UNKNOWN } } }
I20260807 06:33:51.126109 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:51.123398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.2.243.2:36605)
0807 06:33:51.123463 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:51.123465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:51.123475 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:51.123602 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:51.123603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:51.123624 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:51.123716 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:51.123888 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.124373 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.125216 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.125419 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.125496 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:51.125685 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:51.125791 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:51.125929 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:51.125996 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:33:51.126013 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:33:51.127218 15932 raft_consensus.cc:1261] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Refusing update from remote peer 215478c5795744ecbc1229dbda325baf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:51.127244 15448 raft_consensus.cc:1261] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Refusing update from remote peer 215478c5795744ecbc1229dbda325baf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:51.127543 16184 consensus_queue.cc:1035] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:51.127658 16076 consensus_queue.cc:1035] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:51.131057 15166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:51.131157 15166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:51.131773 15166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Sending DeleteTablet for 3 replicas of tablet a2a07edd2e424d3bb6a2bded97991de6
I20260807 06:33:51.132064 15898 tablet_service.cc:1514] Processing DeleteTablet for tablet a2a07edd2e424d3bb6a2bded97991de6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:51 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:40648
I20260807 06:33:51.132097 15663 tablet_service.cc:1514] Processing DeleteTablet for tablet a2a07edd2e424d3bb6a2bded97991de6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:51 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:44384
I20260807 06:33:51.132158 16101 tablet_replica.cc:330] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: stopping tablet replica
I20260807 06:33:51.132189 16102 tablet_replica.cc:330] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: stopping tablet replica
I20260807 06:33:51.132210 16101 raft_consensus.cc:2229] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:51.132237 16102 raft_consensus.cc:2229] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:51.132247 16101 raft_consensus.cc:2258] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:51.132328 16102 raft_consensus.cc:2258] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:51.132560 15420 tablet_service.cc:1514] Processing DeleteTablet for tablet a2a07edd2e424d3bb6a2bded97991de6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:51 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:34254
I20260807 06:33:51.132668 16100 tablet_replica.cc:330] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: stopping tablet replica
I20260807 06:33:51.132731 16100 raft_consensus.cc:2229] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:51.132782 16100 raft_consensus.cc:2258] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:51.132859 16101 ts_tablet_manager.cc:1905] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:51.133158 16102 ts_tablet_manager.cc:1905] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:51.134402 16100 ts_tablet_manager.cc:1905] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:51.134445 15166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 06:33:51.134621 16101 ts_tablet_manager.cc:1918] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 06:33:51.134650 15166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:33:51.134670 16101 log.cc:1196] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/a2a07edd2e424d3bb6a2bded97991de6
I20260807 06:33:51.134893 16101 ts_tablet_manager.cc:1939] T a2a07edd2e424d3bb6a2bded97991de6 P 8ccbba0a2664499886b78b63b40cffc4: Deleting consensus metadata
I20260807 06:33:51.134909 16102 ts_tablet_manager.cc:1918] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:51.134965 16102 log.cc:1196] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/a2a07edd2e424d3bb6a2bded97991de6
I20260807 06:33:51.135200 16102 ts_tablet_manager.cc:1939] T a2a07edd2e424d3bb6a2bded97991de6 P 215478c5795744ecbc1229dbda325baf: Deleting consensus metadata
I20260807 06:33:51.135329 15148 catalog_manager.cc:4841] TS 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873): tablet a2a07edd2e424d3bb6a2bded97991de6 (table test-table [id=e9ed2d16518b47569bf109a3b2494d65]) successfully deleted
I20260807 06:33:51.135522 15148 catalog_manager.cc:4841] TS 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605): tablet a2a07edd2e424d3bb6a2bded97991de6 (table test-table [id=e9ed2d16518b47569bf109a3b2494d65]) successfully deleted
I20260807 06:33:51.136025 16100 ts_tablet_manager.cc:1918] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:33:51.136085 16100 log.cc:1196] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/a2a07edd2e424d3bb6a2bded97991de6
I20260807 06:33:51.136296 16100 ts_tablet_manager.cc:1939] T a2a07edd2e424d3bb6a2bded97991de6 P 8537f495c3d84146b627fcbc8d7ace01: Deleting consensus metadata
I20260807 06:33:51.136686 15146 catalog_manager.cc:4841] TS 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103): tablet a2a07edd2e424d3bb6a2bded97991de6 (table test-table [id=e9ed2d16518b47569bf109a3b2494d65]) successfully deleted
I20260807 06:33:51.136765 15420 tablet_service.cc:1467] Processing CreateTablet for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 (DEFAULT_TABLE table=test-table [id=6372ca9e48854926889e71e4163cd1e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:51.136857 15420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:51.137645 16067 tablet_bootstrap.cc:492] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01: Bootstrap starting.
I20260807 06:33:51.137926 15898 tablet_service.cc:1467] Processing CreateTablet for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 (DEFAULT_TABLE table=test-table [id=6372ca9e48854926889e71e4163cd1e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:51.137951 16067 tablet_bootstrap.cc:654] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:51.138046 15898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:51.138198 15663 tablet_service.cc:1467] Processing CreateTablet for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 (DEFAULT_TABLE table=test-table [id=6372ca9e48854926889e71e4163cd1e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:33:51.138314 15663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:51.139189 16066 tablet_bootstrap.cc:492] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: Bootstrap starting.
I20260807 06:33:51.139257 16071 tablet_bootstrap.cc:492] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: Bootstrap starting.
I20260807 06:33:51.139566 16067 tablet_bootstrap.cc:492] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01: No bootstrap required, opened a new log
I20260807 06:33:51.139585 16071 tablet_bootstrap.cc:654] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:51.139618 16067 ts_tablet_manager.cc:1397] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 06:33:51.139626 16066 tablet_bootstrap.cc:654] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:51.139751 16067 raft_consensus.cc:348] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.139833 16067 raft_consensus.cc:374] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:51.139855 16067 raft_consensus.cc:729] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8537f495c3d84146b627fcbc8d7ace01, State: Initialized, Role: FOLLOWER
I20260807 06:33:51.139906 16067 consensus_queue.cc:260] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.140018 16067 ts_tablet_manager.cc:1428] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:51.140479 16066 tablet_bootstrap.cc:492] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: No bootstrap required, opened a new log
I20260807 06:33:51.140573 16066 ts_tablet_manager.cc:1397] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:51.140704 16071 tablet_bootstrap.cc:492] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: No bootstrap required, opened a new log
I20260807 06:33:51.140760 16071 ts_tablet_manager.cc:1397] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:33:51.140717 16066 raft_consensus.cc:348] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.140831 16066 raft_consensus.cc:374] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:51.140859 16066 raft_consensus.cc:729] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215478c5795744ecbc1229dbda325baf, State: Initialized, Role: FOLLOWER
I20260807 06:33:51.140869 16071 raft_consensus.cc:348] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.140908 16066 consensus_queue.cc:260] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.140944 16071 raft_consensus.cc:374] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:51.140973 16071 raft_consensus.cc:729] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ccbba0a2664499886b78b63b40cffc4, State: Initialized, Role: FOLLOWER
I20260807 06:33:51.141012 16066 ts_tablet_manager.cc:1428] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:51.141029 16071 consensus_queue.cc:260] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.141137 16071 ts_tablet_manager.cc:1428] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:51.255038 16110 raft_consensus.cc:482] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:33:51.255177 16110 raft_consensus.cc:504] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.255442 16110 leader_election.cc:288] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103), 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605)
I20260807 06:33:51.255733 15688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb71645ed6d4c5e9bfe41d04caeb4c2" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf" is_pre_election: true
I20260807 06:33:51.255734 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb71645ed6d4c5e9bfe41d04caeb4c2" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8537f495c3d84146b627fcbc8d7ace01" is_pre_election: true
I20260807 06:33:51.255854 15448 raft_consensus.cc:2454] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 0.
I20260807 06:33:51.255858 15688 raft_consensus.cc:2454] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 0.
I20260807 06:33:51.256057 15818 leader_election.cc:302] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215478c5795744ecbc1229dbda325baf, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:51.256227 16110 raft_consensus.cc:2781] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:33:51.256284 16110 raft_consensus.cc:482] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:33:51.256311 16110 raft_consensus.cc:3037] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:51.257277 16110 raft_consensus.cc:504] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.257419 16110 leader_election.cc:288] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 election: Requested vote from peers 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103), 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605)
I20260807 06:33:51.257623 15688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb71645ed6d4c5e9bfe41d04caeb4c2" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215478c5795744ecbc1229dbda325baf"
I20260807 06:33:51.257603 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb71645ed6d4c5e9bfe41d04caeb4c2" candidate_uuid: "8ccbba0a2664499886b78b63b40cffc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8537f495c3d84146b627fcbc8d7ace01"
I20260807 06:33:51.257710 15688 raft_consensus.cc:3037] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:51.257711 15448 raft_consensus.cc:3037] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:51.258623 15688 raft_consensus.cc:2454] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 1.
I20260807 06:33:51.258682 15448 raft_consensus.cc:2454] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ccbba0a2664499886b78b63b40cffc4 in term 1.
I20260807 06:33:51.258908 15818 leader_election.cc:302] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215478c5795744ecbc1229dbda325baf, 8ccbba0a2664499886b78b63b40cffc4; no voters:
I20260807 06:33:51.259008 16110 raft_consensus.cc:2781] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:51.259073 16110 raft_consensus.cc:686] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 LEADER]: Becoming Leader. State: Replica: 8ccbba0a2664499886b78b63b40cffc4, State: Running, Role: LEADER
I20260807 06:33:51.259131 16110 consensus_queue.cc:237] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } }
I20260807 06:33:51.259707 15163 catalog_manager.cc:5495] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3). New cstate: current_term: 1 leader_uuid: "8ccbba0a2664499886b78b63b40cffc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ccbba0a2664499886b78b63b40cffc4" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42873 } health_report { overall_health: HEALTHY } } }
I20260807 06:33:51.292688 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:51.290538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.2.243.3:42873)
0807 06:33:51.290576 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:51.290577 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:51.290587 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:51.290640 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:51.290640 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:51.290649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:51.290698 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:33:51.290865 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.291332 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.292050 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:51.292246 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:51.292308 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:51.292423 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:33:51.292447 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:33:51.292481 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:51.292578 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 06:33:51.292609 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:33:51.294859 15688 raft_consensus.cc:1261] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Refusing update from remote peer 8ccbba0a2664499886b78b63b40cffc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:51.294900 15448 raft_consensus.cc:1261] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Refusing update from remote peer 8ccbba0a2664499886b78b63b40cffc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:33:51.295085 16110 consensus_queue.cc:1035] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "215478c5795744ecbc1229dbda325baf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:51.295185 16077 consensus_queue.cc:1035] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8537f495c3d84146b627fcbc8d7ace01" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:33:51.305371 15163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:51.305462 15163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:33:51.306130 15163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e337680be7b4f3c906f57147e4a530f: Sending DeleteTablet for 3 replicas of tablet dbb71645ed6d4c5e9bfe41d04caeb4c2
I20260807 06:33:51.306371 15663 tablet_service.cc:1514] Processing DeleteTablet for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:51 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:44384
I20260807 06:33:51.306408 15420 tablet_service.cc:1514] Processing DeleteTablet for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:51 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:34254
I20260807 06:33:51.306406 15898 tablet_service.cc:1514] Processing DeleteTablet for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:33:51 UTC) from {username='oryx', principal='oryx/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:40648
I20260807 06:33:51.306545 16101 tablet_replica.cc:330] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: stopping tablet replica
I20260807 06:33:51.306569 16100 tablet_replica.cc:330] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01: stopping tablet replica
I20260807 06:33:51.306515 16102 tablet_replica.cc:330] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: stopping tablet replica
I20260807 06:33:51.306605 16101 raft_consensus.cc:2229] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:33:51.306628 16100 raft_consensus.cc:2229] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:51.306627 16102 raft_consensus.cc:2229] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:33:51.306659 16101 raft_consensus.cc:2258] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:51.306670 16102 raft_consensus.cc:2258] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:51.306666 16100 raft_consensus.cc:2258] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:33:51.306989 16100 ts_tablet_manager.cc:1905] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8537f495c3d84146b627fcbc8d7ace01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:51.307083 16102 ts_tablet_manager.cc:1905] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:51.307161 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 15295
I20260807 06:33:51.310099 16102 ts_tablet_manager.cc:1918] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:51.310161 16102 log.cc:1196] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/dbb71645ed6d4c5e9bfe41d04caeb4c2
I20260807 06:33:51.310184 16101 ts_tablet_manager.cc:1905] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:33:51.310400 16102 ts_tablet_manager.cc:1939] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 215478c5795744ecbc1229dbda325baf: Deleting consensus metadata
I20260807 06:33:51.310770 15148 catalog_manager.cc:4841] TS 215478c5795744ecbc1229dbda325baf (127.2.243.2:36605): tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 (table test-table [id=6372ca9e48854926889e71e4163cd1e8]) successfully deleted
I20260807 06:33:51.311956 16101 ts_tablet_manager.cc:1918] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:33:51.312022 16101 log.cc:1196] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/dbb71645ed6d4c5e9bfe41d04caeb4c2
I20260807 06:33:51.312209 16101 ts_tablet_manager.cc:1939] T dbb71645ed6d4c5e9bfe41d04caeb4c2 P 8ccbba0a2664499886b78b63b40cffc4: Deleting consensus metadata
I20260807 06:33:51.314613 15148 catalog_manager.cc:4841] TS 8ccbba0a2664499886b78b63b40cffc4 (127.2.243.3:42873): tablet dbb71645ed6d4c5e9bfe41d04caeb4c2 (table test-table [id=6372ca9e48854926889e71e4163cd1e8]) successfully deleted
W20260807 06:33:51.325757 15146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:39103 (error 108)
I20260807 06:33:51.326013 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 15548
I20260807 06:33:51.330652 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:51.326181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39103 (local address 127.0.0.1:34274)
0807 06:33:51.330520 (+ 4339us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:51.330597 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4315,"thread_start_us":41,"threads_started":1}
W20260807 06:33:51.332055 15146 catalog_manager.cc:4584] TS 8537f495c3d84146b627fcbc8d7ace01 (127.2.243.1:39103): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbb71645ed6d4c5e9bfe41d04caeb4c2: Network error: Client connection negotiation failed: client connection to 127.2.243.1:39103: connect: Connection refused (error 111)
I20260807 06:33:51.339597 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 15787
I20260807 06:33:51.352476 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 15111
2026-08-07T06:33:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3259 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:51 dist-test-slave-rwrg krb5kdc[16215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:51 dist-test-slave-rwrg krb5kdc[16215](info): set up 2 sockets
Aug 07 06:33:51 dist-test-slave-rwrg krb5kdc[16215](info): commencing operation
krb5kdc: starting...
W20260807 06:33:52.620075 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:52 dist-test-slave-rwrg krb5kdc[16215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:52.747176 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:36437
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:36437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:52.829334 16233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:52.829571 16233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:52.829609 16233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:52.830742 16233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:52.830812 16233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:52.830837 16233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:52.830868 16233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:52.830899 16233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:52.832427 16233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:36437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:36437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:52.832726 16233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:52.832962 16233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:52.837921 16248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:52.837972 16242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:52.838145 16233 server_base.cc:1034] running on GCE node
W20260807 06:33:52.837942 16243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:52.838451 16233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:52.838768 16233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:52.839955 16233 hybrid_clock.cc:648] HybridClock initialized: now 1786084432839934 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T06:33:52Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1486 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:33:52 dist-test-slave-rwrg krb5kdc[16253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:33:52 dist-test-slave-rwrg krb5kdc[16253](info): set up 2 sockets
Aug 07 06:33:52 dist-test-slave-rwrg krb5kdc[16253](info): commencing operation
krb5kdc: starting...
W20260807 06:33:54.139001 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:33:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:33:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:33:54.262987 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42099
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:54.344276 16275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:54.344513 16275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:54.344538 16275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:54.345623 16275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:54.345660 16275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:54.345674 16275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:54.345686 16275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:54.345697 16275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:54.347138 16275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:54.347386 16275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:54.347611 16275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:54.352427 16291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:54.352448 16283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:54.352417 16289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:54.352648 16275 server_base.cc:1034] running on GCE node
I20260807 06:33:54.352864 16275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:54.353119 16275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:54.354279 16275 hybrid_clock.cc:648] HybridClock initialized: now 1786084434354261 us; error 37 us; skew 500 ppm
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:54.361624 16275 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:54.362030 16275 webserver.cc:460] Webserver started at http://127.2.243.62:43789/ using document root <none> and password file <none>
I20260807 06:33:54.362218 16275 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:54.362257 16275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:54.362375 16275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:54.363469 16275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "ad34c4cc601c4afaa69b2f019e08e290"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "adf8cfe4590f8439d74346ee7b3ece91"
server_key_iv: "77b845479582a3f6e31c3f5371602e99"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.363775 16275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "ad34c4cc601c4afaa69b2f019e08e290"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "adf8cfe4590f8439d74346ee7b3ece91"
server_key_iv: "77b845479582a3f6e31c3f5371602e99"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.365161 16275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:33:54.365932 16304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.366250 16275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:54.366346 16275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "ad34c4cc601c4afaa69b2f019e08e290"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "adf8cfe4590f8439d74346ee7b3ece91"
server_key_iv: "77b845479582a3f6e31c3f5371602e99"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.366405 16275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:54.390687 16275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:54.392654 16275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:54.392786 16275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:54.396699 16275 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42099
I20260807 06:33:54.396756 16394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42099 every 8 connection(s)
I20260807 06:33:54.397109 16275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:54.398070 16400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:33:54.398229 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 16275
I20260807 06:33:54.398362 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:33:54.398578 3020 external_mini_cluster.cc:1428] Setting key 87d2e5ce7325ae13fd696cc45114e4bb
I20260807 06:33:54.401374 16400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Bootstrap starting.
I20260807 06:33:54.402164 16400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Neither blocks nor log segments found. Creating new log.
I20260807 06:33:54.402463 16400 log.cc:824] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:54.403214 16400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: No bootstrap required, opened a new log
I20260807 06:33:54.405346 16400 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:54.405544 16400 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:33:54.405596 16400 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad34c4cc601c4afaa69b2f019e08e290, State: Initialized, Role: FOLLOWER
I20260807 06:33:54.405730 16400 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:54.405822 16400 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:54.405866 16400 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:54.405911 16400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:33:54.406631 16400 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:54.406770 16400 leader_election.cc:302] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad34c4cc601c4afaa69b2f019e08e290; no voters:
I20260807 06:33:54.406951 16400 leader_election.cc:288] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:33:54.407021 16410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:33:54.407207 16410 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 LEADER]: Becoming Leader. State: Replica: ad34c4cc601c4afaa69b2f019e08e290, State: Running, Role: LEADER
I20260807 06:33:54.407250 16400 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:54.408023 16410 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:54.408706 16411 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad34c4cc601c4afaa69b2f019e08e290. Latest consensus state: current_term: 1 leader_uuid: "ad34c4cc601c4afaa69b2f019e08e290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } } }
I20260807 06:33:54.409039 16411 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:54.408849 16412 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad34c4cc601c4afaa69b2f019e08e290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } } }
I20260807 06:33:54.409284 16412 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: This master's current role is: LEADER
W20260807 06:33:54.409291 16427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:33:54.409435 16427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:33:54.410351 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:54.399141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.2.243.62:42099)
0807 06:33:54.399338 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:54.399348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:54.399553 (+ 205us) server_negotiation.cc:407] Connection header received
0807 06:33:54.400155 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:54.400164 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:54.400223 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:54.400375 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:54.401077 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.402380 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.403190 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.403681 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.403803 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:54.406451 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:54.406455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:54.406455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:54.406466 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:54.408678 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.409123 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.409126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.409126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.409244 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.409488 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.409488 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.409489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.409731 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:54.409841 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:54.410009 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 06:33:54.410117 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260807 06:33:54.412649 16432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:54.412873 16432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:54.414861 16432 catalog_manager.cc:1347] Generated new cluster ID: 0cc8772673304f4a80b601537733bce4
I20260807 06:33:54.414920 16432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:54.427249 16432 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:33:54.427877 16432 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:54.433799 16432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Generated new TSK 0
I20260807 06:33:54.434003 16432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:33:54.471077 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42099
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:54.553618 16445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:54.553813 16445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:54.553836 16445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:54.554908 16445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:54.554940 16445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:54.554965 16445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:54.556380 16445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:54.556625 16445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:54.556798 16445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:54.561493 16452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:54.561681 16445 server_base.cc:1034] running on GCE node
W20260807 06:33:54.561575 16456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:54.561494 16450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:54.562007 16445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:54.562258 16445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:54.563428 16445 hybrid_clock.cc:648] HybridClock initialized: now 1786084434563419 us; error 38 us; skew 500 ppm
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:54.566150 16445 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:33:54.566598 16445 webserver.cc:460] Webserver started at http://127.2.243.1:40303/ using document root <none> and password file <none>
I20260807 06:33:54.566841 16445 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:54.566908 16445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:54.567070 16445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:54.568065 16445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "b821719d36fa4263a351c17fd08e7d31"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "95fe03cbaa92c8fc32676ecbf13e3dc6"
server_key_iv: "1a27cff02cc5aed5c1c6d8847453aba2"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.568418 16445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "b821719d36fa4263a351c17fd08e7d31"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "95fe03cbaa92c8fc32676ecbf13e3dc6"
server_key_iv: "1a27cff02cc5aed5c1c6d8847453aba2"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.569904 16445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:54.570711 16470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.570897 16445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.570987 16445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "b821719d36fa4263a351c17fd08e7d31"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "95fe03cbaa92c8fc32676ecbf13e3dc6"
server_key_iv: "1a27cff02cc5aed5c1c6d8847453aba2"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.571071 16445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:54.581969 16445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:54.583801 16445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:54.583971 16445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:54.584209 16445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:54.584684 16445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:54.584743 16445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.584779 16445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:54.584805 16445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.590627 16445 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40091
I20260807 06:33:54.590679 16668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40091 every 8 connection(s)
I20260807 06:33:54.590969 16445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:33:54.596465 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 16445
I20260807 06:33:54.596668 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:54.596865 3020 external_mini_cluster.cc:1428] Setting key bfd429e180b8e2d6184d44e1db1417ec
I20260807 06:33:54.600479 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:54.591665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:33923 (local address 127.2.243.62:42099)
0807 06:33:54.591754 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:54.591756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:54.591941 (+ 185us) server_negotiation.cc:407] Connection header received
0807 06:33:54.592386 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:54.592387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:54.592398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:54.592452 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:54.593439 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.593927 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.594854 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.595052 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.595102 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:54.597426 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:54.597428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:54.597428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:54.597437 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:54.598999 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.599507 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.599508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.599508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.599576 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.599858 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.599858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.599859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.599957 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:54.600005 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:54.600335 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 06:33:54.600418 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:33:54.600805 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:54.591632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:33923)
0807 06:33:54.591820 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:54.591854 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:54.592302 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:54.592577 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:54.592581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:54.593171 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:54.593352 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:54.593356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.594015 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.594016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:54.594780 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:54.594782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.595123 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.595124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:54.595219 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:54.595673 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:54.595680 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:54.597336 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:54.599095 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:54.599101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:54.599103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:54.599422 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:54.599647 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:54.599648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:54.599649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:54.599793 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:54.600019 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:54.600024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:54.600194 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 06:33:54.600317 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 06:33:54.600544 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260807 06:33:54.601348 16670 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42099
I20260807 06:33:54.601462 16670 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:54.601684 16670 heartbeater.cc:511] Master 127.2.243.62:42099 requested a full tablet report, sending...
I20260807 06:33:54.602270 16339 ts_manager.cc:194] Registered new tserver with Master: b821719d36fa4263a351c17fd08e7d31 (127.2.243.1:40091)
I20260807 06:33:54.602849 16339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:33923
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:33:54.625258 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42099
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:54.710554 16678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:54.710773 16678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:54.710796 16678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:54.711935 16678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:54.711975 16678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:54.712006 16678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:54.713508 16678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:54.713762 16678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:54.713999 16678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:54.718968 16684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:54.718999 16688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:54.718978 16686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:54.719235 16678 server_base.cc:1034] running on GCE node
I20260807 06:33:54.719465 16678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:54.719700 16678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:54.720867 16678 hybrid_clock.cc:648] HybridClock initialized: now 1786084434720845 us; error 39 us; skew 500 ppm
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:54.723304 16678 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:33:54.723680 16678 webserver.cc:460] Webserver started at http://127.2.243.2:45407/ using document root <none> and password file <none>
I20260807 06:33:54.723932 16678 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:54.723978 16678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:54.724175 16678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:54.725344 16678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "f74be8c11058481685b65429a4009f86"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "e8d1c72af32b4beb00d9d4a4f83664ad"
server_key_iv: "962a8e15e0d61a1c27ebb3ae22fe6997"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.725711 16678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "f74be8c11058481685b65429a4009f86"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "e8d1c72af32b4beb00d9d4a4f83664ad"
server_key_iv: "962a8e15e0d61a1c27ebb3ae22fe6997"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.727069 16678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:33:54.727857 16707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.728078 16678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:54.728173 16678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "f74be8c11058481685b65429a4009f86"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "e8d1c72af32b4beb00d9d4a4f83664ad"
server_key_iv: "962a8e15e0d61a1c27ebb3ae22fe6997"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.728248 16678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:54.739972 16678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:54.742411 16678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:54.742579 16678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:54.742803 16678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:54.743125 16678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:54.743181 16678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.743223 16678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:54.743250 16678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.749174 16678 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:39271
I20260807 06:33:54.749259 16895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:39271 every 8 connection(s)
I20260807 06:33:54.749590 16678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:54.751178 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 16678
I20260807 06:33:54.751314 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:33:54.751609 3020 external_mini_cluster.cc:1428] Setting key c2fbed00d90161c12af3fe8ed21c4e87
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:54.759876 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:54.750379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:48529 (local address 127.2.243.62:42099)
0807 06:33:54.750462 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:54.750464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:54.750572 (+ 108us) server_negotiation.cc:407] Connection header received
0807 06:33:54.751063 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:54.751064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:54.751079 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:54.751154 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:54.752311 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.753083 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.754120 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.754337 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.754394 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:54.756952 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:54.756953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:54.756954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:54.756961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:54.758439 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.758870 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.758871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.758871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.758935 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.759210 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.759211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.759212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.759332 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:54.759413 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:54.759702 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 06:33:54.759784 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:33:54.760074 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:54.750285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:48529)
0807 06:33:54.750439 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:54.750484 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:54.750968 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:54.751221 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:54.751224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:54.752023 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:54.752213 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:54.752218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.753172 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.753173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:54.754014 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:54.754018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.754422 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.754423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:54.754575 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:54.755074 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:54.755086 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:54.756857 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:54.758548 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:54.758556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:54.758558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:54.758792 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:54.759023 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:54.759024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:54.759024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:54.759139 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:54.759404 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:54.759411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:54.759560 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 06:33:54.759666 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:33:54.759827 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
I20260807 06:33:54.760586 16897 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42099
I20260807 06:33:54.760684 16897 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:54.760877 16897 heartbeater.cc:511] Master 127.2.243.62:42099 requested a full tablet report, sending...
I20260807 06:33:54.761292 16339 ts_manager.cc:194] Registered new tserver with Master: f74be8c11058481685b65429a4009f86 (127.2.243.2:39271)
I20260807 06:33:54.761765 16339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:48529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:33:54.777935 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42099
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:54.860546 16906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:54.860752 16906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:54.860775 16906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:54.861955 16906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:54.861999 16906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:54.862026 16906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:54.863463 16906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:54.863705 16906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:54.863946 16906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:54.869256 16916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:54.869303 16920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:54.869372 16906 server_base.cc:1034] running on GCE node
W20260807 06:33:54.869318 16918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:54.869730 16906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:54.870051 16906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:54.871210 16906 hybrid_clock.cc:648] HybridClock initialized: now 1786084434871199 us; error 32 us; skew 500 ppm
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:54.874068 16906 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:33:54.874466 16906 webserver.cc:460] Webserver started at http://127.2.243.3:35331/ using document root <none> and password file <none>
I20260807 06:33:54.874648 16906 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:54.874689 16906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:54.874776 16906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:33:54.875777 16906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "5e2c1ac6f7cd4fc9b7029c7510369277"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "78f7a7e307f59bb66407ed3c40f31e25"
server_key_iv: "8ca81e8f5df5ffd89889b430c8b133e8"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.876086 16906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "5e2c1ac6f7cd4fc9b7029c7510369277"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "78f7a7e307f59bb66407ed3c40f31e25"
server_key_iv: "8ca81e8f5df5ffd89889b430c8b133e8"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.877437 16906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:33:54.878147 16929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.878305 16906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:33:54.878397 16906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "5e2c1ac6f7cd4fc9b7029c7510369277"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "78f7a7e307f59bb66407ed3c40f31e25"
server_key_iv: "8ca81e8f5df5ffd89889b430c8b133e8"
server_key_version: "encryptionkey@0"
I20260807 06:33:54.878450 16906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:54.890254 16906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:54.892316 16906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:54.892441 16906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:54.892745 16906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:54.893039 16906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:54.893074 16906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.893096 16906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:54.893110 16906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:54.898881 16906 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:42457
I20260807 06:33:54.898960 17109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:42457 every 8 connection(s)
I20260807 06:33:54.899272 16906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:33:54.903451 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 16906
I20260807 06:33:54.903590 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:33:54.903802 3020 external_mini_cluster.cc:1428] Setting key 52dd8dc92ddfb19c4e2dc7166ad9340f
Aug 07 06:33:54 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084434, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:33:54.909384 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:54.900185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:45413 (local address 127.2.243.62:42099)
0807 06:33:54.900249 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:54.900251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:54.900453 (+ 202us) server_negotiation.cc:407] Connection header received
0807 06:33:54.900937 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:54.900938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:54.900948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:54.901007 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:54.901955 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.902435 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.903735 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.903965 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.904026 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:54.906348 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:54.906350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:54.906350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:54.906356 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:54.907815 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.908258 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.908258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.908259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.908308 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:54.908679 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:54.908680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:54.908680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:54.908787 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:54.908888 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:54.909183 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 06:33:54.909275 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:33:54.909576 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:54.900146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:45413)
0807 06:33:54.900316 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:54.900359 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:54.900855 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:54.901072 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:54.901076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:54.901667 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:54.901865 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:54.901870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.902556 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.902556 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:54.903649 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:54.903652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:54.904060 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:54.904062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:54.904187 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:54.904828 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:54.904839 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:54.906265 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:33:54.907892 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:54.907898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:54.907899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:54.908194 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:54.908459 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:33:54.908460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:33:54.908460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:33:54.908614 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:33:54.908882 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:33:54.908889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:33:54.909041 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 06:33:54.909149 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 06:33:54.909328 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260807 06:33:54.910102 17110 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42099
I20260807 06:33:54.910209 17110 heartbeater.cc:464] Registering TS with master...
I20260807 06:33:54.910403 17110 heartbeater.cc:511] Master 127.2.243.62:42099 requested a full tablet report, sending...
I20260807 06:33:54.910787 16339 ts_manager.cc:194] Registered new tserver with Master: 5e2c1ac6f7cd4fc9b7029c7510369277 (127.2.243.3:42457)
I20260807 06:33:54.911156 16339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:45413
I20260807 06:33:54.915814 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab.
Entry for principal oryx/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab.
WARNING: no policy specified for oryx/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab.
Entry for principal oryx/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab.
WARNING: no policy specified for oryx/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab.
Entry for principal oryx/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab.
I20260807 06:33:54.991389 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 16445
I20260807 06:33:55.002472 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 16678
I20260807 06:33:55.016791 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 16906
I20260807 06:33:55.027520 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 16275
I20260807 06:33:55.047951 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42099
--webserver_interface=127.2.243.62
--webserver_port=43789
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:33:55.129242 17123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:55.129467 17123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:55.129488 17123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:55.130663 17123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:33:55.130707 17123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:55.130721 17123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:55.130734 17123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:33:55.130785 17123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:33:55.132282 17123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=43789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:55.132594 17123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:55.132846 17123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:55.137761 17138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:55.137755 17133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:55.137892 17132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:55.137763 17123 server_base.cc:1034] running on GCE node
I20260807 06:33:55.138200 17123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:55.138438 17123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:55.139612 17123 hybrid_clock.cc:648] HybridClock initialized: now 1786084435139585 us; error 40 us; skew 500 ppm
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:55.142359 17123 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:33:55.142874 17123 webserver.cc:460] Webserver started at http://127.2.243.62:43789/ using document root <none> and password file <none>
I20260807 06:33:55.143124 17123 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:55.143184 17123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:55.144734 17123 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:55.145448 17147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.145689 17123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:55.145793 17123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "ad34c4cc601c4afaa69b2f019e08e290"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "adf8cfe4590f8439d74346ee7b3ece91"
server_key_iv: "77b845479582a3f6e31c3f5371602e99"
server_key_version: "encryptionkey@0"
I20260807 06:33:55.146137 17123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:55.162470 17123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:55.164610 17123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:55.164752 17123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:55.169772 17123 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42099
I20260807 06:33:55.169811 17234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42099 every 8 connection(s)
I20260807 06:33:55.170229 17123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:33:55.171743 17237 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:33:55.172473 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Bootstrap starting.
I20260807 06:33:55.173282 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 17123
I20260807 06:33:55.173693 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:40091
--local_ip_for_outbound_sockets=127.2.243.1
--tserver_master_addrs=127.2.243.62:42099
--webserver_port=40303
--webserver_interface=127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:33:55.181627 17237 log.cc:824] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Log is configured to *not* fsync() on all Append() calls
I20260807 06:33:55.183512 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:33:55.183816 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Bootstrap complete.
I20260807 06:33:55.186173 17237 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:55.186437 17237 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad34c4cc601c4afaa69b2f019e08e290, State: Initialized, Role: FOLLOWER
I20260807 06:33:55.186645 17237 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:55.186761 17237 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:33:55.186816 17237 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:33:55.186864 17237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:33:55.187855 17237 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:55.187979 17237 leader_election.cc:302] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad34c4cc601c4afaa69b2f019e08e290; no voters:
I20260807 06:33:55.188180 17237 leader_election.cc:288] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:33:55.188216 17243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:33:55.188338 17243 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [term 2 LEADER]: Becoming Leader. State: Replica: ad34c4cc601c4afaa69b2f019e08e290, State: Running, Role: LEADER
I20260807 06:33:55.188462 17237 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:33:55.188510 17243 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } }
I20260807 06:33:55.188885 17246 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad34c4cc601c4afaa69b2f019e08e290. Latest consensus state: current_term: 2 leader_uuid: "ad34c4cc601c4afaa69b2f019e08e290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } } }
I20260807 06:33:55.188977 17246 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:55.189139 17244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ad34c4cc601c4afaa69b2f019e08e290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad34c4cc601c4afaa69b2f019e08e290" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42099 } } }
I20260807 06:33:55.189189 17244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290 [sys.catalog]: This master's current role is: LEADER
I20260807 06:33:55.189721 17251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:33:55.190300 17251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:33:55.190519 17251 catalog_manager.cc:1259] Loaded cluster ID: 0cc8772673304f4a80b601537733bce4
I20260807 06:33:55.190593 17251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:33:55.191857 17251 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:33:55.192077 17251 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad34c4cc601c4afaa69b2f019e08e290: Loaded TSK: 0
I20260807 06:33:55.192307 17251 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:33:55.258735 17241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:55.258971 17241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:55.259008 17241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:55.260228 17241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:55.260299 17241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:55.260344 17241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:33:55.261823 17241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.1.keytab
--principal=oryx/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:40091
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=40303
--tserver_master_addrs=127.2.243.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:55.262089 17241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:55.262315 17241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:55.268620 17274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:55.268625 17272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:55.268640 17269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:55.268909 17241 server_base.cc:1034] running on GCE node
I20260807 06:33:55.269155 17241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:55.269438 17241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:55.270603 17241 hybrid_clock.cc:648] HybridClock initialized: now 1786084435270581 us; error 43 us; skew 500 ppm
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084435, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:55.273563 17241 init.cc:377] Logged in from keytab as oryx/127.2.243.1@KRBTEST.COM (short username oryx)
I20260807 06:33:55.274040 17241 webserver.cc:460] Webserver started at http://127.2.243.1:40303/ using document root <none> and password file <none>
I20260807 06:33:55.274281 17241 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:55.274339 17241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:55.275713 17241 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:55.276372 17285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.276563 17241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:55.276670 17241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "b821719d36fa4263a351c17fd08e7d31"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "95fe03cbaa92c8fc32676ecbf13e3dc6"
server_key_iv: "1a27cff02cc5aed5c1c6d8847453aba2"
server_key_version: "encryptionkey@0"
I20260807 06:33:55.276993 17241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:55.290422 17241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:55.292881 17241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:55.293052 17241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:55.293311 17241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:55.293695 17241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:55.293738 17241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.293761 17241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:55.293773 17241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.301095 17241 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40091
I20260807 06:33:55.301134 17484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40091 every 8 connection(s)
I20260807 06:33:55.301476 17241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.310076 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 17241
I20260807 06:33:55.310454 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:39271
--local_ip_for_outbound_sockets=127.2.243.2
--tserver_master_addrs=127.2.243.62:42099
--webserver_port=45407
--webserver_interface=127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.311762 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.302238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:45759)
0807 06:33:55.302505 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.302546 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.303242 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.303700 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.303704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.304584 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.304870 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.304877 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.306003 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.306004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.307331 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.307334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.307912 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.307913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.308057 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.308696 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.308706 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.311555 (+ 2849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":99,"threads_started":1}
W20260807 06:33:55.311942 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.302290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:45759 (local address 127.2.243.62:42099)
0807 06:33:55.302462 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.302471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.302654 (+ 183us) server_negotiation.cc:407] Connection header received
0807 06:33:55.303419 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.303425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.303463 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.303606 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.305007 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.305894 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.307424 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.307803 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.307918 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.311719 (+ 3801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:45759: BlockingRecv error: recv got EOF from 127.2.243.1:45759 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
W20260807 06:33:55.312397 17485 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:42099 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.243.62:42099: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.318461 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.312661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:54757)
0807 06:33:55.312772 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.312783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.312829 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.312980 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.312980 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.313148 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.313203 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.313205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.314097 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.314098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.314980 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.314982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.315504 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.315505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.315584 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.316110 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.316116 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.318368 (+ 2252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260807 06:33:55.318557 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.312723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:54757 (local address 127.2.243.62:42099)
0807 06:33:55.312768 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.312770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.312816 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:33:55.312880 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.312881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.312894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.312963 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.313286 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.313986 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.315075 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.315410 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.315492 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.318482 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:54757: BlockingRecv error: recv got EOF from 127.2.243.1:54757 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.324687 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.319101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:51563)
0807 06:33:55.319231 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.319245 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.319326 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.319491 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.319493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.319709 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.319740 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.319742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.320685 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.320686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.321574 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.321577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.322097 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.322098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.322154 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.322621 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.322627 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.324613 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110}
W20260807 06:33:55.324769 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.319177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:51563 (local address 127.2.243.62:42099)
0807 06:33:55.319242 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.319244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.319307 (+ 63us) server_negotiation.cc:407] Connection header received
0807 06:33:55.319398 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.319399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.319412 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.319475 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.319824 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.320594 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.321672 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.321999 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.322084 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.324710 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:51563: BlockingRecv error: recv got EOF from 127.2.243.1:51563 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 06:33:55.325018 17485 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:33:55.397768 17492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:55.397998 17492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:55.398036 17492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:55.399159 17492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:55.399226 17492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:55.399268 17492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:33:55.400803 17492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.2.keytab
--principal=oryx/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:39271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=45407
--tserver_master_addrs=127.2.243.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:55.401068 17492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:55.401299 17492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:55.407213 17500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:55.407253 17504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:55.407210 17501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:55.407305 17492 server_base.cc:1034] running on GCE node
I20260807 06:33:55.407577 17492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:55.407810 17492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:55.408952 17492 hybrid_clock.cc:648] HybridClock initialized: now 1786084435408933 us; error 32 us; skew 500 ppm
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084435, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:55.411381 17492 init.cc:377] Logged in from keytab as oryx/127.2.243.2@KRBTEST.COM (short username oryx)
I20260807 06:33:55.411868 17492 webserver.cc:460] Webserver started at http://127.2.243.2:45407/ using document root <none> and password file <none>
I20260807 06:33:55.412101 17492 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:55.412215 17492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:55.413688 17492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:55.414290 17519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.414458 17492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:55.414568 17492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "f74be8c11058481685b65429a4009f86"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "e8d1c72af32b4beb00d9d4a4f83664ad"
server_key_iv: "962a8e15e0d61a1c27ebb3ae22fe6997"
server_key_version: "encryptionkey@0"
I20260807 06:33:55.414899 17492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:55.445855 17492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:55.448077 17492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:55.448308 17492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:55.448633 17492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:55.448956 17492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:55.449013 17492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.449056 17492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:55.449083 17492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.454814 17492 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:39271
I20260807 06:33:55.454859 17718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:39271 every 8 connection(s)
I20260807 06:33:55.455215 17492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:33:55.457042 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 17492
I20260807 06:33:55.457433 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:42457
--local_ip_for_outbound_sockets=127.2.243.3
--tserver_master_addrs=127.2.243.62:42099
--webserver_port=35331
--webserver_interface=127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.463757 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.455921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:52843)
0807 06:33:55.456113 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.456149 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.456705 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.456984 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.456993 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.457721 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.457915 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.457921 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.458620 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.458621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.459925 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.459929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.460292 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.460294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.460442 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.461032 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.461042 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.463553 (+ 2511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
W20260807 06:33:55.463762 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.455935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:52843 (local address 127.2.243.62:42099)
0807 06:33:55.456015 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.456017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.456238 (+ 221us) server_negotiation.cc:407] Connection header received
0807 06:33:55.456818 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.456819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.456834 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.456911 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.458023 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.458522 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.460025 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.460214 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.460277 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.463674 (+ 3397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:52843: BlockingRecv error: recv got EOF from 127.2.243.2:52843 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 06:33:55.464087 17719 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:42099 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.243.62:42099: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.468804 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.464342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:41409)
0807 06:33:55.464384 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.464393 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.464448 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.464594 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.464595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.464729 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.464773 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.464775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.465405 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.465406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.465934 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.465936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.466316 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.466317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.466388 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.466867 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.466871 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.468759 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 06:33:55.468953 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.464404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:41409 (local address 127.2.243.62:42099)
0807 06:33:55.464468 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.464471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.464479 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:55.464517 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.464518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.464528 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.464589 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.464862 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.465323 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.466032 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.466229 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.466306 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.468870 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:41409: BlockingRecv error: recv got EOF from 127.2.243.2:41409 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.473794 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.469351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:43401)
0807 06:33:55.469469 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.469481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.469533 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.469665 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.469666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.469805 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.469838 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.469839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.470411 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.470412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.471228 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.471231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.471537 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.471538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.471593 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.471984 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.471989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.473722 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260807 06:33:55.473863 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.469419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:43401 (local address 127.2.243.62:42099)
0807 06:33:55.469474 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.469476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.469519 (+ 43us) server_negotiation.cc:407] Connection header received
0807 06:33:55.469588 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.469589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.469596 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.469642 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.469902 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.470340 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.471306 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.471469 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.471516 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.473808 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:43401: BlockingRecv error: recv got EOF from 127.2.243.2:43401 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 06:33:55.474045 17719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:33:55.543049 17731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:33:55.543306 17731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:33:55.543335 17731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:33:55.544421 17731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:33:55.544457 17731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:33:55.544538 17731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:33:55.545930 17731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/krb5kdc/oryx_127.2.243.3.keytab
--principal=oryx/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:42457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=35331
--tserver_master_addrs=127.2.243.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:33:55.546110 17731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:33:55.546277 17731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:33:55.551506 17749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:33:55.551478 17746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:55.551623 17731 server_base.cc:1034] running on GCE node
W20260807 06:33:55.551478 17745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:33:55.552033 17731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:33:55.552281 17731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:33:55.553436 17731 hybrid_clock.cc:648] HybridClock initialized: now 1786084435553403 us; error 54 us; skew 500 ppm
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084435, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:33:55.556066 17731 init.cc:377] Logged in from keytab as oryx/127.2.243.3@KRBTEST.COM (short username oryx)
I20260807 06:33:55.556456 17731 webserver.cc:460] Webserver started at http://127.2.243.3:35331/ using document root <none> and password file <none>
I20260807 06:33:55.556708 17731 fs_manager.cc:359] Metadata directory not provided
I20260807 06:33:55.556766 17731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:33:55.558082 17731 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:33:55.558630 17760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.558792 17731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:33:55.558871 17731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "5e2c1ac6f7cd4fc9b7029c7510369277"
format_stamp: "Formatted at 2026-08-07 06:33:54 on dist-test-slave-rwrg"
server_key: "78f7a7e307f59bb66407ed3c40f31e25"
server_key_iv: "8ca81e8f5df5ffd89889b430c8b133e8"
server_key_version: "encryptionkey@0"
I20260807 06:33:55.559185 17731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:33:55.565866 17731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:33:55.567788 17731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:33:55.567902 17731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:33:55.568084 17731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:33:55.568332 17731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:33:55.568365 17731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.568387 17731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:33:55.568399 17731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:33:55.574352 17731 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:42457
I20260807 06:33:55.574415 17940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:42457 every 8 connection(s)
I20260807 06:33:55.574774 17731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:33:55.581845 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.575551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:52453 (local address 127.2.243.62:42099)
0807 06:33:55.575629 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.575632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.575777 (+ 145us) server_negotiation.cc:407] Connection header received
0807 06:33:55.576251 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.576252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.576261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.576317 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.577287 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.577768 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.578681 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.578851 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.578901 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.581799 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:52453: BlockingRecv error: recv got EOF from 127.2.243.3:52453 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:33:55.581856 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.575473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:52453)
0807 06:33:55.575641 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.575682 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.576134 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.576374 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.576378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.577013 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.577201 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.577205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.577849 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.577850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.578607 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.578608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.578922 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.578923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.579016 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.579523 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.579531 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.581709 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
W20260807 06:33:55.582119 17942 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:42099 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.243.62:42099: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
I20260807 06:33:55.583894 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 17731
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.587621 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.582360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:45477)
0807 06:33:55.582410 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.582423 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.582466 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.582835 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.582836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.583011 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.583082 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.583084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.583757 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.583759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.584755 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.584758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.585240 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.585241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.585312 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.585790 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.585795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.587575 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:33:55.587733 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.582658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:45477 (local address 127.2.243.62:42099)
0807 06:33:55.582708 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.582710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.582718 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:55.582739 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.582740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.582756 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.582804 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.583174 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.583639 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.584850 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.585150 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.585228 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.587659 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:45477: BlockingRecv error: recv got EOF from 127.2.243.3:45477 (error 108)
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":31}
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:55 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:55.592532 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.587931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:37563)
0807 06:33:55.587983 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:55.587993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:55.588034 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:55.588186 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:55.588187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:55.588313 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:55.588337 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.588338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.589064 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.589065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.589905 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:55.589908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.590319 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.590320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:55.590390 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.590669 (+ 279us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:55.590676 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:55.592445 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 06:33:55.592607 17491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:55.588039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:37563 (local address 127.2.243.62:42099)
0807 06:33:55.588082 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.588084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.588092 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:55.588112 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.588113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.588122 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.588178 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.588414 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.588956 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.589991 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.590237 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.590305 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.592563 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:37563: BlockingRecv error: recv got EOF from 127.2.243.3:37563 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:33:55.593003 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:55.584328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.2.243.62:42099)
0807 06:33:55.584937 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:55.584939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:55.584947 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:33:55.584973 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:55.584974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:55.584987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:55.585055 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:55.585814 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.586326 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.587297 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:55.587515 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:55.587591 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:55.589224 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:33:55.589229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:33:55.589229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:33:55.589242 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:33:55.591321 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:55.591723 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:55.591727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:55.591727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:55.591872 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:33:55.592086 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:33:55.592087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:33:55.592087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:33:55.592356 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:33:55.592495 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 06:33:55.592752 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 06:33:55.592904 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":553,"threads_started":1}
W20260807 06:33:55.592996 17942 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 06:33:56 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:56 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:56.330718 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:56.325448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:50505)
0807 06:33:56.325648 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:56.325659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:56.325712 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:56.325857 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:56.325858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:56.326026 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:56.326075 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:56.326076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.326806 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.326807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:56.327589 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:56.327592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.328009 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.328010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:56.328063 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:56.328640 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:56.328644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:56.330635 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":90,"threads_started":1}
W20260807 06:33:56.330852 17950 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:56.325549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:50505 (local address 127.2.243.62:42099)
0807 06:33:56.325714 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:56.325716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:56.325730 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:33:56.325758 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:56.325758 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:56.325769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:56.325835 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:56.326148 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.326655 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.327689 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.327889 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.328017 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:56.330744 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:50505: BlockingRecv error: recv got EOF from 127.2.243.1:50505 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
Aug 07 06:33:56 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:56 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:56.479269 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:56.474469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:58439)
0807 06:33:56.474606 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:56.474624 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:56.474684 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:56.474812 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:56.474813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:56.474975 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:56.475025 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:56.475027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.475749 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.475750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:56.476320 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:56.476322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.476782 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.476783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:56.476835 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:56.477319 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:56.477324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:56.479195 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
W20260807 06:33:56.479363 17950 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:56.474558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:58439 (local address 127.2.243.62:42099)
0807 06:33:56.474613 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:56.474615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:56.474686 (+ 71us) server_negotiation.cc:407] Connection header received
0807 06:33:56.474730 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:56.474730 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:56.474741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:56.474795 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:56.475130 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.475661 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.476416 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.476703 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.476766 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:56.479297 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:58439: BlockingRecv error: recv got EOF from 127.2.243.2:58439 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 06:33:56 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:56 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:56.598091 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:56.593390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:45627)
0807 06:33:56.593534 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:56.593549 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:56.593618 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:56.593755 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:56.593757 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:56.593925 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:56.593973 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:56.593975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.594662 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.594663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:56.595210 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:56.595212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.595577 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.595578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:56.595637 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:56.596116 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:56.596121 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:56.598014 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
W20260807 06:33:56.598196 17950 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:56.593460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:45627 (local address 127.2.243.62:42099)
0807 06:33:56.593587 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:56.593589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:56.593599 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:56.593681 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:56.593682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:56.593695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:56.593735 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:56.594041 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.594552 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.595308 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:56.595492 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:56.595556 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:56.598119 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:45627: BlockingRecv error: recv got EOF from 127.2.243.3:45627 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 06:33:57 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:57 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:57.336311 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:57.331247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:58433)
0807 06:33:57.331422 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:57.331433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:57.331522 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:57.331651 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:57.331652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:57.331817 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:57.331884 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:57.331886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.332678 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.332679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:57.333332 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:57.333335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.333719 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.333720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:57.333776 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:57.334222 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:57.334226 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:57.336202 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
W20260807 06:33:57.336555 17958 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:57.331350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:58433 (local address 127.2.243.62:42099)
0807 06:33:57.331487 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:57.331489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:57.331506 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:33:57.331562 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:57.331563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:57.331572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:57.331634 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:57.331952 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.332595 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.333417 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.333639 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.333719 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:57.336332 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:58433: BlockingRecv error: recv got EOF from 127.2.243.1:58433 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
Aug 07 06:33:57 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:57 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:57.484886 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:57.479820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:37859)
0807 06:33:57.479986 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:57.479997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:57.480077 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:57.480181 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:57.480182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:57.480356 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:57.480413 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:57.480415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.481199 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.481200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:57.481945 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:57.481948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.482299 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.482300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:57.482340 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:57.482798 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:57.482803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:57.484783 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
W20260807 06:33:57.484941 17958 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:57.479928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:37859 (local address 127.2.243.62:42099)
0807 06:33:57.479989 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:57.479992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:57.480075 (+ 83us) server_negotiation.cc:407] Connection header received
0807 06:33:57.480120 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:57.480120 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:57.480131 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:57.480178 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:57.480537 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.481106 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.482043 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.482229 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.482286 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:57.484875 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:37859: BlockingRecv error: recv got EOF from 127.2.243.2:37859 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 06:33:57 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:57 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:57.603410 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:57.598675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:41427)
0807 06:33:57.598853 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:57.598866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:57.598929 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:57.599068 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:57.599068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:57.599231 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:57.599277 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:57.599278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.599941 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.599942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:57.600475 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:57.600477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.600852 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.600853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:57.600893 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:57.601363 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:57.601367 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:57.603295 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
W20260807 06:33:57.603451 17958 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:57.598836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:41427 (local address 127.2.243.62:42099)
0807 06:33:57.598905 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:57.598907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:57.598918 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:57.598981 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:57.598982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:57.598995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:57.599049 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:57.599345 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.599863 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.600583 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:57.600781 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:57.600843 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:57.603381 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:41427: BlockingRecv error: recv got EOF from 127.2.243.3:41427 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 06:33:58 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:58 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:58.341843 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:58.336955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:50943)
0807 06:33:58.337129 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:58.337141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:58.337203 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:58.337428 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:58.337429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:58.337644 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:58.337694 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:58.337696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.338392 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.338393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:58.338938 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:58.338940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.339315 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.339316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:58.339373 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:58.339846 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:58.339850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:58.341736 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
W20260807 06:33:58.341892 17970 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:58.337097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:50943 (local address 127.2.243.62:42099)
0807 06:33:58.337234 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:58.337236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:58.337246 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:33:58.337270 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:58.337270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:58.337281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:58.337380 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:58.337788 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.338301 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.339034 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.339238 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.339317 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:58.341818 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:50943: BlockingRecv error: recv got EOF from 127.2.243.1:50943 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
Aug 07 06:33:58 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:58 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:58.490507 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:58.485474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:52017)
0807 06:33:58.485656 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:58.485668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:58.485764 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:58.485920 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:58.485921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:58.486106 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:58.486153 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:58.486155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.486810 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.486811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:58.487339 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:58.487341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.487702 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.487703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:58.487760 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:58.488225 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:58.488230 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:58.490396 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
W20260807 06:33:58.490562 17970 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:58.485569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:52017 (local address 127.2.243.62:42099)
0807 06:33:58.485640 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:58.485643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:58.485742 (+ 99us) server_negotiation.cc:407] Connection header received
0807 06:33:58.485832 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:58.485833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:58.485846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:58.485914 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:58.486243 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.486727 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.487425 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.487626 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.487704 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:58.490497 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:52017: BlockingRecv error: recv got EOF from 127.2.243.2:52017 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 06:33:58 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:58 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:58.609900 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:58.604053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:46651)
0807 06:33:58.604212 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:58.604228 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:58.604285 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:58.604524 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:58.604526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:58.604719 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:58.604802 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:58.604804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.605506 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.605508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:58.606467 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:58.606471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.607019 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.607020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:58.607094 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:58.607590 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:58.607595 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:58.609821 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
W20260807 06:33:58.610005 17970 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:58.604192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:46651 (local address 127.2.243.62:42099)
0807 06:33:58.604259 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:58.604266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:58.604277 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:33:58.604365 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:58.604365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:58.604379 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:58.604426 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:58.604898 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.605411 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.606570 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:58.606910 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:58.607001 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:58.609912 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:46651: BlockingRecv error: recv got EOF from 127.2.243.3:46651 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 06:33:59 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:59 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:59.347539 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:59.342437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:47841)
0807 06:33:59.342699 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:59.342713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:59.342778 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:59.343017 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:59.343018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:59.343254 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:59.343303 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:59.343305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.344022 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.344023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:59.344583 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:59.344585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.344972 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.344972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:59.345011 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:59.345559 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:59.345564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:59.347444 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":85,"threads_started":1}
W20260807 06:33:59.347601 17975 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:59.342532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:47841 (local address 127.2.243.62:42099)
0807 06:33:59.342691 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:59.342694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:59.342787 (+ 93us) server_negotiation.cc:407] Connection header received
0807 06:33:59.342870 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:59.342885 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:59.342902 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:59.342981 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:59.343409 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.343943 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.344678 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.344901 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.344964 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:59.347534 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:47841: BlockingRecv error: recv got EOF from 127.2.243.1:47841 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
Aug 07 06:33:59 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:59 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:59.504576 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:59.491143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:42625)
0807 06:33:59.491294 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:59.491304 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:59.491353 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:59.491685 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:59.491686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:59.491853 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:59.491903 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:59.491905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.492736 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.492737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:59.493285 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:59.493287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.493691 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.493692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:59.493734 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:59.502547 (+ 8813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:59.502553 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:59.504439 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
W20260807 06:33:59.504714 17975 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:59.491353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:42625 (local address 127.2.243.62:42099)
0807 06:33:59.491506 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:59.491508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:59.491536 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:33:59.491577 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:59.491577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:59.491590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:59.491656 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:59.491987 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.492628 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.493385 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.493611 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.493677 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:59.504617 (+ 10940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:42625: BlockingRecv error: recv got EOF from 127.2.243.2:42625 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 06:33:59 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:33:59 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:33:59.615185 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:33:59.610396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:53155)
0807 06:33:59.610579 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 06:33:59.610593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:33:59.610677 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:33:59.610832 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:33:59.610833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:33:59.611014 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:33:59.611059 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:59.611061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.611736 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.611738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:59.612279 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:33:59.612280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.612692 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.612693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:33:59.612756 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:59.613228 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:33:59.613233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:33:59.615125 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
W20260807 06:33:59.615317 17975 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:33:59.610509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:53155 (local address 127.2.243.62:42099)
0807 06:33:59.610597 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 06:33:59.610600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:33:59.610678 (+ 78us) server_negotiation.cc:407] Connection header received
0807 06:33:59.610735 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:33:59.610736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:33:59.610752 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:33:59.610825 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:33:59.611143 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.611644 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.612366 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:33:59.612602 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:33:59.612691 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:33:59.615234 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:53155: BlockingRecv error: recv got EOF from 127.2.243.3:53155 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 06:34:00 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:00 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:00.353559 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:00.348130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:35249)
0807 06:34:00.348301 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:00.348322 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:00.348376 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:00.348633 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:00.348634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:00.348864 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:00.348917 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:00.348920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.349884 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.349885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:00.350431 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:00.350432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.350837 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.350838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:00.350883 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:00.351402 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:00.351407 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:00.353462 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":124,"threads_started":1}
W20260807 06:34:00.353768 17981 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:00.348272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:35249 (local address 127.2.243.62:42099)
0807 06:34:00.348410 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:00.348411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:00.348420 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:34:00.348442 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:00.348442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:00.348453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:00.348620 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:00.349023 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.349746 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.350562 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.350760 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.350826 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:00.353671 (+ 2845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:35249: BlockingRecv error: recv got EOF from 127.2.243.1:35249 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
Aug 07 06:34:00 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:00 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:00.509927 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:00.505109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:38993)
0807 06:34:00.505289 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:00.505300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:00.505361 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:00.505489 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:00.505490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:00.505664 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:00.505713 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:00.505714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.506341 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.506342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:00.506856 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:00.506857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.507185 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.507186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:00.507228 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:00.507719 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:00.507723 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:00.509868 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
W20260807 06:34:00.510077 17981 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:00.505296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:38993 (local address 127.2.243.62:42099)
0807 06:34:00.505364 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:00.505366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:00.505392 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:34:00.505416 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:00.505417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:00.505428 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:00.505467 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:00.505788 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.506273 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.506929 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.507116 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.507168 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:00.509985 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:38993: BlockingRecv error: recv got EOF from 127.2.243.2:38993 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 06:34:00 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:00 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:00.620575 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:00.615731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:50449)
0807 06:34:00.615903 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:00.615914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:00.615996 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:00.616141 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:00.616142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:00.616306 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:00.616354 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:00.616355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.617058 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.617059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:00.617724 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:00.617727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.618085 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.618086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:00.618142 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:00.618637 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:00.618641 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:00.620450 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
W20260807 06:34:00.620622 17981 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:00.615805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:50449 (local address 127.2.243.62:42099)
0807 06:34:00.615876 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:00.615878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:00.615992 (+ 114us) server_negotiation.cc:407] Connection header received
0807 06:34:00.616053 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:00.616054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:00.616068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:00.616117 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:00.616414 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.616962 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.617796 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:00.618011 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:00.618085 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:00.620560 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:50449: BlockingRecv error: recv got EOF from 127.2.243.3:50449 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 06:34:01 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:01 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:01.358747 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:01.354034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:49071)
0807 06:34:01.354181 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:01.354209 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:01.354275 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:01.354529 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:01.354530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:01.354705 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:01.354754 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:01.354756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.355490 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.355490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:01.356015 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:01.356016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.356391 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.356392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:01.356435 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:01.356968 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:01.356973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:01.358689 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
W20260807 06:34:01.358891 17988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:01.354235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:49071 (local address 127.2.243.62:42099)
0807 06:34:01.354385 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:01.354387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:01.354395 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:34:01.354419 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:01.354420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:01.354431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:01.354513 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:01.354841 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.355414 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.356108 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.356313 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.356378 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:01.358797 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:49071: BlockingRecv error: recv got EOF from 127.2.243.1:49071 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
Aug 07 06:34:01 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:01 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:01.514995 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:01.510416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:39503)
0807 06:34:01.510549 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:01.510561 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:01.510656 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:01.510858 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:01.510859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:01.511041 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:01.511090 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:01.511092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.511719 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.511720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:01.512220 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:01.512221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.512578 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.512579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:01.512637 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:01.513093 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:01.513097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:01.514939 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260807 06:34:01.515121 17988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:01.510492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:39503 (local address 127.2.243.62:42099)
0807 06:34:01.510543 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:01.510546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:01.510662 (+ 116us) server_negotiation.cc:407] Connection header received
0807 06:34:01.510732 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:01.510733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:01.510749 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:01.510830 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:01.511158 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.511645 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.512297 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.512479 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.512577 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:01.515036 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:39503: BlockingRecv error: recv got EOF from 127.2.243.2:39503 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 07 06:34:01 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:01 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:01.625837 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:01.621091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:33509)
0807 06:34:01.621265 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:01.621275 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:01.621333 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:01.621478 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:01.621479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:01.621671 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:01.621721 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:01.621722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.622380 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.622381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:01.622949 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:01.622950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.623301 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.623302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:01.623351 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:01.623810 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:01.623815 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:01.625778 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
W20260807 06:34:01.625996 17988 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:01.621185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:33509 (local address 127.2.243.62:42099)
0807 06:34:01.621277 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:01.621279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:01.621321 (+ 42us) server_negotiation.cc:407] Connection header received
0807 06:34:01.621402 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:01.621403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:01.621416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:01.621459 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:01.621814 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.622295 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.623035 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:01.623227 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:01.623285 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:01.625916 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:33509: BlockingRecv error: recv got EOF from 127.2.243.3:33509 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 06:34:02 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:02 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:02.364261 17999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:02.359313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:50953)
0807 06:34:02.359475 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:02.359488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:02.359569 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:02.359771 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:02.359773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:02.359978 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:02.360029 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:02.360031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.360716 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.360717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:02.361269 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:02.361271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.361726 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.361727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:02.361788 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:02.362233 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:02.362238 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:02.364148 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
W20260807 06:34:02.364323 18001 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:02.359414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:50953 (local address 127.2.243.62:42099)
0807 06:34:02.359554 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:02.359556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:02.359566 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:34:02.359656 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:02.359657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:02.359670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:02.359750 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:02.360112 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.360622 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.361351 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.361647 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.361712 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:02.364234 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:50953: BlockingRecv error: recv got EOF from 127.2.243.1:50953 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
Aug 07 06:34:02 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:02 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:02.520439 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:02.515473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:38489)
0807 06:34:02.515669 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:02.515682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:02.515772 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:02.515964 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:02.515965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:02.516147 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:02.516195 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:02.516197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.516978 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.516980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:02.517537 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:02.517539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.517916 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.517917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:02.517984 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:02.518470 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:02.518475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:02.520381 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":108,"threads_started":1}
W20260807 06:34:02.520591 18001 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:02.515574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:38489 (local address 127.2.243.62:42099)
0807 06:34:02.515662 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:02.515665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:02.515752 (+ 87us) server_negotiation.cc:407] Connection header received
0807 06:34:02.515850 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:02.515851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:02.515864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:02.515944 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:02.516278 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.516855 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.517632 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.517821 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.517886 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:02.520476 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:38489: BlockingRecv error: recv got EOF from 127.2.243.2:38489 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 06:34:02 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:02 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:02.631140 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:02.626303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:33373)
0807 06:34:02.626445 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:02.626456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:02.626514 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:02.626715 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:02.626717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:02.626889 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:02.626939 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:02.626941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.627576 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.627577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:02.628103 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:02.628105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.628420 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.628421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:02.628476 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:02.629120 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:02.629125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:02.631081 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
W20260807 06:34:02.631259 18001 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:02.626407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:33373 (local address 127.2.243.62:42099)
0807 06:34:02.626456 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:02.626458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:02.626500 (+ 42us) server_negotiation.cc:407] Connection header received
0807 06:34:02.626564 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:02.626565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:02.626603 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:02.626657 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:02.627023 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.627502 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.628179 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:02.628350 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:02.628400 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:02.631180 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:33373: BlockingRecv error: recv got EOF from 127.2.243.3:33373 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 06:34:03 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:03 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:03.369503 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:03.364773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:50837)
0807 06:34:03.364946 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:03.364957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:03.365039 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:03.365197 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:03.365198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:03.365360 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:03.365409 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:03.365410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.366061 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.366062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:03.366550 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:03.366552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.366873 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.366874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:03.366913 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:03.367363 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:03.367368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:03.369443 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
W20260807 06:34:03.369632 18005 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:03.364894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:50837 (local address 127.2.243.62:42099)
0807 06:34:03.365047 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:03.365049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:03.365060 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:34:03.365105 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:03.365106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:03.365116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:03.365175 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:03.365476 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.365972 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.366620 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.366791 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.366859 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:03.369550 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:50837: BlockingRecv error: recv got EOF from 127.2.243.1:50837 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
Aug 07 06:34:03 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:03 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:03.525923 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:03.520948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:34195)
0807 06:34:03.521137 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:03.521155 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:03.521222 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:03.521377 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:03.521378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:03.521596 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:03.521647 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:03.521649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.522415 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.522416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:03.522959 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:03.522960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.523279 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.523280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:03.523319 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:03.523793 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:03.523798 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:03.525857 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":134,"threads_started":1}
W20260807 06:34:03.526047 18005 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:03.521007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:34195 (local address 127.2.243.62:42099)
0807 06:34:03.521108 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:03.521112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:03.521246 (+ 134us) server_negotiation.cc:407] Connection header received
0807 06:34:03.521280 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:03.521280 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:03.521294 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:03.521354 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:03.521720 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.522347 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.523036 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.523213 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.523269 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:03.525963 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:34195: BlockingRecv error: recv got EOF from 127.2.243.2:34195 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 06:34:03 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:03 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:03.636960 18007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:03.631670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:36191)
0807 06:34:03.631967 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:03.631977 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:03.632066 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:03.632221 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:03.632222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:03.632420 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:03.632471 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:03.632473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.633414 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.633415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:03.633953 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:03.633954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.634337 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.634338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:03.634397 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:03.634926 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:03.634931 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:03.636900 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":237,"threads_started":1}
W20260807 06:34:03.637151 18005 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:03.631790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:36191 (local address 127.2.243.62:42099)
0807 06:34:03.631857 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:03.631859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:03.632057 (+ 198us) server_negotiation.cc:407] Connection header received
0807 06:34:03.632125 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:03.632126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:03.632142 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:03.632215 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:03.632635 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.633327 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.634043 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:03.634246 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:03.634336 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:03.637066 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:36191: BlockingRecv error: recv got EOF from 127.2.243.3:36191 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 06:34:04 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:04 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:04.375983 18011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:04.370118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:36901)
0807 06:34:04.370337 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:04.370352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:04.370459 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:04.370681 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:04.370683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:04.370979 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:04.371047 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:04.371050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.371866 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.371867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:04.372488 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:04.372518 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.372998 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.372999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:04.373067 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:04.373782 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:04.373787 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:04.375868 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":132,"threads_started":1}
W20260807 06:34:04.376065 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:04.370272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:36901 (local address 127.2.243.62:42099)
0807 06:34:04.370462 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:04.370467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:04.370515 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:34:04.370552 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:04.370553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:04.370565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:04.370652 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:04.371143 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.371730 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.372627 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.372859 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.373000 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:04.375970 (+ 2970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:36901: BlockingRecv error: recv got EOF from 127.2.243.1:36901 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
Aug 07 06:34:04 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:04 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:04.537829 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:04.526677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:44905)
0807 06:34:04.526953 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:04.526966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:04.527064 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:04.527362 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:04.527363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:04.527738 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:04.527864 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:04.527866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.529053 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.529055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:04.529969 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:04.529972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.530556 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.530557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:04.530651 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:04.535555 (+ 4904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:04.535561 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:04.537712 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":173,"threads_started":1}
W20260807 06:34:04.537948 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:04.526821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:44905 (local address 127.2.243.62:42099)
0807 06:34:04.526942 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:04.526946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:04.527046 (+ 100us) server_negotiation.cc:407] Connection header received
0807 06:34:04.527182 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:04.527183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:04.527207 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:04.527303 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:04.527978 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.528892 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.530116 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.530406 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.530524 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:04.537865 (+ 7341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:44905: BlockingRecv error: recv got EOF from 127.2.243.2:44905 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 06:34:04 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:04 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:04.642341 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:04.637455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:48093)
0807 06:34:04.637693 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:04.637706 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:04.637769 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:04.637895 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:04.637895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:04.638073 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:04.638121 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:04.638122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.638773 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.638774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:04.639305 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:04.639307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.639706 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.639708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:04.639765 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:04.640291 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:04.640296 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:04.642279 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":160,"threads_started":1}
W20260807 06:34:04.642453 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:04.637526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:48093 (local address 127.2.243.62:42099)
0807 06:34:04.637630 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:04.637632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:04.637772 (+ 140us) server_negotiation.cc:407] Connection header received
0807 06:34:04.637821 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:04.637822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:04.637835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:04.637876 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:04.638188 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.638690 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.639394 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:04.639590 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:04.639648 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:04.642377 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:48093: BlockingRecv error: recv got EOF from 127.2.243.3:48093 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 06:34:05 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:05 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:05.381724 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:05.376645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:45651)
0807 06:34:05.376839 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:05.376851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:05.376919 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:05.377075 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:05.377076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:05.377258 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:05.377340 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:05.377342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.378101 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.378102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:05.378911 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:05.378915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.379240 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.379241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:05.379283 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:05.379756 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:05.379761 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:05.381665 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
W20260807 06:34:05.381850 18021 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:05.376792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:45651 (local address 127.2.243.62:42099)
0807 06:34:05.376912 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:05.376913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:05.376923 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:34:05.376979 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:05.376980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:05.376990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:05.377052 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:05.377411 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.377956 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.378982 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.379185 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.379241 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:05.381763 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:45651: BlockingRecv error: recv got EOF from 127.2.243.1:45651 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
Aug 07 06:34:05 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:05 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:05.543532 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:05.538375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:51587)
0807 06:34:05.538623 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:05.538637 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:05.538744 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:05.538953 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:05.538954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:05.539168 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:05.539220 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:05.539222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.539989 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.539991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:05.540572 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:05.540575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.540986 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.540987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:05.541047 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:05.541588 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:05.541593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:05.543473 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":158,"threads_started":1}
W20260807 06:34:05.543752 18021 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:05.538460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:51587 (local address 127.2.243.62:42099)
0807 06:34:05.538605 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:05.538607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:05.538722 (+ 115us) server_negotiation.cc:407] Connection header received
0807 06:34:05.538846 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:05.538847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:05.538870 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:05.538944 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:05.539310 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.539891 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.540686 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.540896 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.540985 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:05.543671 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:51587: BlockingRecv error: recv got EOF from 127.2.243.2:51587 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 07 06:34:05 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:05 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:05.647854 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:05.642861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:42223)
0807 06:34:05.643010 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:05.643020 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:05.643070 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:05.643219 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:05.643219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:05.643390 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:05.643437 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:05.643439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.644097 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.644098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:05.644681 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:05.644682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.645077 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.645078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:05.645116 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:05.645640 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:05.645645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:05.647743 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
W20260807 06:34:05.647981 18021 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:05.642994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:42223 (local address 127.2.243.62:42099)
0807 06:34:05.643076 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:05.643080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:05.643114 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:34:05.643137 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:05.643137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:05.643150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:05.643195 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:05.643518 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.644019 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.644784 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:05.645002 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:05.645080 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:05.647895 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:42223: BlockingRecv error: recv got EOF from 127.2.243.3:42223 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 06:34:06 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:06 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:06.386924 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:06.382303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:34707)
0807 06:34:06.382444 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:06.382466 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:06.382524 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:06.382690 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:06.382691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:06.382866 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:06.382911 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:06.382912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.383572 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.383573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:06.384103 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:06.384105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.384431 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.384432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:06.384470 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:06.384941 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:06.384946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:06.386852 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
W20260807 06:34:06.387084 18029 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:06.382385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:34707 (local address 127.2.243.62:42099)
0807 06:34:06.382541 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:06.382543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:06.382583 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:34:06.382606 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:06.382606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:06.382617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:06.382671 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:06.383002 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.383498 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.384178 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.384365 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.384417 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:06.386977 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:34707: BlockingRecv error: recv got EOF from 127.2.243.1:34707 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":143,"threads_started":1}
Aug 07 06:34:06 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:06 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:06.548903 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:06.544034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:48839)
0807 06:34:06.544191 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:06.544205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:06.544277 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:06.544459 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:06.544460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:06.544681 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:06.544727 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:06.544729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.545416 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.545417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:06.545969 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:06.545971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.546338 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.546339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:06.546407 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:06.546887 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:06.546892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:06.548787 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
W20260807 06:34:06.548936 18029 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:06.544150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:48839 (local address 127.2.243.62:42099)
0807 06:34:06.544230 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:06.544232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:06.544258 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:34:06.544376 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:06.544377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:06.544389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:06.544449 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:06.544823 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.545321 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.546055 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.546265 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.546339 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:06.548867 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:48839: BlockingRecv error: recv got EOF from 127.2.243.2:48839 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 06:34:06 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:06 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:06.653479 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:06.648476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:34451)
0807 06:34:06.648699 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:06.648712 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:06.648780 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:06.648922 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:06.648923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:06.649099 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:06.649149 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:06.649151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.650039 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.650041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:06.650598 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:06.650599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.650924 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.650924 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:06.650965 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:06.651423 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:06.651427 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:06.653359 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":121,"threads_started":1}
W20260807 06:34:06.653520 18029 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:06.648721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:34451 (local address 127.2.243.62:42099)
0807 06:34:06.648787 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:06.648789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:06.648800 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:34:06.648831 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:06.648831 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:06.648843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:06.648895 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:06.649270 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.649947 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.650676 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:06.650857 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:06.650908 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:06.653449 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:34451: BlockingRecv error: recv got EOF from 127.2.243.3:34451 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 06:34:07 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:07 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:07.393196 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:07.387597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:39485)
0807 06:34:07.387832 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:07.387849 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:07.387936 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:07.388138 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:07.388139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:07.388324 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:07.388373 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:07.388375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.389589 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.389590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:07.390213 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:07.390215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.390632 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.390633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:07.390674 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:07.391196 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:07.391200 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:07.393127 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":123,"threads_started":1}
W20260807 06:34:07.393376 18040 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:07.387713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:39485 (local address 127.2.243.62:42099)
0807 06:34:07.387850 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:07.387858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:07.387931 (+ 73us) server_negotiation.cc:407] Connection header received
0807 06:34:07.387995 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:07.387996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:07.388009 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:07.388080 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:07.388551 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.389465 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.390299 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.390551 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.390631 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:07.393254 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:39485: BlockingRecv error: recv got EOF from 127.2.243.1:39485 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
Aug 07 06:34:07 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:07 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:07.554299 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:07.549463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:55319)
0807 06:34:07.549655 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:07.549669 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:07.549741 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:07.549869 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:07.549870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:07.550099 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:07.550148 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:07.550150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.550793 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.550794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:07.551322 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:07.551325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.551708 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.551709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:07.551747 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:07.552185 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:07.552191 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:07.554194 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
W20260807 06:34:07.554343 18040 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:07.549596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:55319 (local address 127.2.243.62:42099)
0807 06:34:07.549675 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:07.549677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:07.549740 (+ 63us) server_negotiation.cc:407] Connection header received
0807 06:34:07.549785 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:07.549786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:07.549796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:07.549854 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:07.550212 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.550712 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.551418 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.551626 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.551700 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:07.554269 (+ 2569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:55319: BlockingRecv error: recv got EOF from 127.2.243.2:55319 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 06:34:07 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:07 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:07.658876 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:07.653996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:34121)
0807 06:34:07.654149 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:07.654169 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:07.654225 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:07.654351 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:07.654352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:07.654561 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:07.654608 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:07.654609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.655248 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.655248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:07.655855 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:07.655857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.656312 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.656313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:07.656361 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:07.656953 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:07.656958 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:07.658802 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
W20260807 06:34:07.659016 18040 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:07.654073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:34121 (local address 127.2.243.62:42099)
0807 06:34:07.654146 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:07.654148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:07.654224 (+ 76us) server_negotiation.cc:407] Connection header received
0807 06:34:07.654267 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:07.654267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:07.654280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:07.654332 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:07.654675 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.655167 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.656033 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:07.656232 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:07.656296 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:07.658901 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:34121: BlockingRecv error: recv got EOF from 127.2.243.3:34121 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 06:34:08 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:08 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:08.398229 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:08.393697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:36971)
0807 06:34:08.393824 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:08.393838 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:08.393900 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:08.394035 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:08.394035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:08.394216 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:08.394263 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:08.394265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.394889 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.394889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:08.395414 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:08.395415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.395765 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.395767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:08.395811 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:08.396262 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:08.396266 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:08.398158 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
W20260807 06:34:08.398362 18049 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:08.393761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:36971 (local address 127.2.243.62:42099)
0807 06:34:08.393890 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:08.393892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:08.393907 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:34:08.393946 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:08.393946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:08.393957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:08.394015 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:08.394331 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.394824 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.395482 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.395695 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.395747 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:08.398254 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:36971: BlockingRecv error: recv got EOF from 127.2.243.1:36971 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
Aug 07 06:34:08 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:08 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:08.559733 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:08.554935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:60075)
0807 06:34:08.555137 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:08.555149 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:08.555206 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:08.555338 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:08.555339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:08.555539 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:08.555585 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:08.555586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.556231 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.556233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:08.556961 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:08.556965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.557298 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.557299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:08.557335 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:08.557827 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:08.557832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:08.559664 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":149,"threads_started":1}
W20260807 06:34:08.559928 18049 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:08.555070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:60075 (local address 127.2.243.62:42099)
0807 06:34:08.555139 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:08.555141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:08.555206 (+ 65us) server_negotiation.cc:407] Connection header received
0807 06:34:08.555251 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:08.555252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:08.555265 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:08.555321 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:08.555658 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.556176 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.557039 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.557233 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.557290 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:08.559842 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:60075: BlockingRecv error: recv got EOF from 127.2.243.2:60075 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 06:34:08 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:08 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:08.665220 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:08.659387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:37233)
0807 06:34:08.659645 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:08.659660 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:08.659767 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:08.659984 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:08.659985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:08.660201 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:08.660252 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:08.660254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.661128 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.661129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:08.661762 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:08.661765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.662263 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.662264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:08.662333 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:08.662949 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:08.662955 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:08.665108 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":146,"threads_started":1}
W20260807 06:34:08.665395 18049 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:08.659488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:37233 (local address 127.2.243.62:42099)
0807 06:34:08.659629 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:08.659632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:08.659746 (+ 114us) server_negotiation.cc:407] Connection header received
0807 06:34:08.659853 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:08.659854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:08.659873 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:08.659960 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:08.660347 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.661018 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.661865 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:08.662161 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:08.662260 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:08.665236 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:37233: BlockingRecv error: recv got EOF from 127.2.243.3:37233 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 06:34:09 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:09 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:09.403774 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:09.398716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:37711)
0807 06:34:09.398878 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:09.398890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:09.398954 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:09.399130 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:09.399131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:09.399324 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:09.399376 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:09.399378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.400025 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.400026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:09.400650 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:09.400652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.400990 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.400991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:09.401038 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:09.401652 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:09.401657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:09.403707 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260807 06:34:09.403858 18056 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:09.398857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:37711 (local address 127.2.243.62:42099)
0807 06:34:09.399004 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:09.399006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:09.399013 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:34:09.399034 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:09.399035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:09.399045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:09.399107 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:09.399452 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.399949 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.400726 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.400910 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.400967 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:09.403801 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:37711: BlockingRecv error: recv got EOF from 127.2.243.1:37711 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
Aug 07 06:34:09 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:09 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:09.569016 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:09.560306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:48937)
0807 06:34:09.560578 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:09.560589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:09.560675 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:09.560805 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:09.560806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:09.561034 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:09.561079 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:09.561082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.561795 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.561796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:09.562355 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:09.562357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.562747 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.562748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:09.562817 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:09.567085 (+ 4268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:09.567090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:09.568943 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":122,"threads_started":1}
W20260807 06:34:09.569116 18056 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:09.560410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:48937 (local address 127.2.243.62:42099)
0807 06:34:09.560511 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:09.560514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:09.560661 (+ 147us) server_negotiation.cc:407] Connection header received
0807 06:34:09.560737 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:09.560738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:09.560750 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:09.560801 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:09.561161 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.561709 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.562443 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.562648 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.562729 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:09.569050 (+ 6321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:48937: BlockingRecv error: recv got EOF from 127.2.243.2:48937 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 06:34:09 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:09 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.3@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:09.670621 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:09.665739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.3:40503)
0807 06:34:09.665926 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:09.665935 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:09.666009 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:09.666205 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:09.666206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:09.666414 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:09.666460 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:09.666462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.667156 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.667158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:09.667687 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:09.667689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.668094 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.668095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:09.668158 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:09.668718 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:09.668722 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:09.670515 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
W20260807 06:34:09.670678 18056 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:09.665798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:40503 (local address 127.2.243.62:42099)
0807 06:34:09.665868 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:09.665870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:09.665996 (+ 126us) server_negotiation.cc:407] Connection header received
0807 06:34:09.666094 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:09.666094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:09.666110 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:09.666181 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:09.666546 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.667062 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.667776 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:09.668000 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:09.668092 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:09.670612 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.3:40503: BlockingRecv error: recv got EOF from 127.2.243.3:40503 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 06:34:10 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:10 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.1@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:10.409128 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:10.404373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.1:38153)
0807 06:34:10.404586 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:10.404600 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:10.404671 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:10.404831 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:10.404832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:10.404997 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:10.405048 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:10.405050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.405704 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.405705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:10.406237 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:10.406238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.406590 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.406591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:10.406631 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:10.407090 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:10.407095 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:10.409058 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
W20260807 06:34:10.409245 18063 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:10.404434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:38153 (local address 127.2.243.62:42099)
0807 06:34:10.404591 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:10.404593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:10.404677 (+ 84us) server_negotiation.cc:407] Connection header received
0807 06:34:10.404731 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:10.404732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:10.404743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:10.404813 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:10.405121 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.405625 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.406318 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.406524 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.406579 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:10.409155 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.1:38153: BlockingRecv error: recv got EOF from 127.2.243.1:38153 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":133,"threads_started":1}
Aug 07 06:34:10 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:34:10 dist-test-slave-rwrg krb5kdc[16253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.243.2@KRBTEST.COM for oryx/127.2.243.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:34:10.574296 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:10.569567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42099 (local address 127.2.243.2:58321)
0807 06:34:10.569772 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:10.569787 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:10.569866 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:10.570025 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:10.570026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:10.570219 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:10.570268 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:10.570269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.570951 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.570952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:10.571512 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:10.571514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.571915 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.571916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:10.571977 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:10.572443 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:10.572448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:10.574237 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.243.62:42099: Server oryx/127.2.243.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
W20260807 06:34:10.574416 18063 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:10.569676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:58321 (local address 127.2.243.62:42099)
0807 06:34:10.569776 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:10.569778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:10.569863 (+ 85us) server_negotiation.cc:407] Connection header received
0807 06:34:10.569928 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:10.569928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:10.569942 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:10.570016 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:10.570355 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.570853 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.571634 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.571836 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.571916 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:10.574339 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.243.2:58321: BlockingRecv error: recv got EOF from 127.2.243.2:58321 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 06:34:10.586910 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 17241
I20260807 06:34:10.600714 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 17492
I20260807 06:34:10.611954 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 17731
I20260807 06:34:10.620997 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 17123
2026-08-07T06:34:10Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17782 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T06:34:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:34:10Z Disabled control of system clock
I20260807 06:34:10.657037 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45157
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:45157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 06:34:10.739979 18071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:10.740223 18071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:10.740258 18071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:10.741503 18071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:34:10.741587 18071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:10.741609 18071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:10.741628 18071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:34:10.741665 18071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:34:10.743053 18071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:45157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:10.743348 18071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:10.743662 18071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:10.748915 18079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:10.748987 18080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:10.749085 18071 server_base.cc:1034] running on GCE node
W20260807 06:34:10.748935 18082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:10.749504 18071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:10.749837 18071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:10.751001 18071 hybrid_clock.cc:648] HybridClock initialized: now 1786084450750979 us; error 52 us; skew 500 ppm
I20260807 06:34:10.752269 18071 webserver.cc:460] Webserver started at http://127.2.243.62:35627/ using document root <none> and password file <none>
I20260807 06:34:10.752593 18071 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:10.752672 18071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:10.752812 18071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:10.753866 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "01e45312631348d3979e93c53366012d"
format_stamp: "Formatted at 2026-08-07 06:34:10 on dist-test-slave-rwrg"
server_key: "a6d4329c95602175542270a52eebff6b"
server_key_iv: "3541f05fd31c8dbd5cf694fc9d945cfb"
server_key_version: "encryptionkey@0"
I20260807 06:34:10.754253 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "01e45312631348d3979e93c53366012d"
format_stamp: "Formatted at 2026-08-07 06:34:10 on dist-test-slave-rwrg"
server_key: "a6d4329c95602175542270a52eebff6b"
server_key_iv: "3541f05fd31c8dbd5cf694fc9d945cfb"
server_key_version: "encryptionkey@0"
I20260807 06:34:10.755594 18071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:34:10.756315 18094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:10.756526 18071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:34:10.756636 18071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "01e45312631348d3979e93c53366012d"
format_stamp: "Formatted at 2026-08-07 06:34:10 on dist-test-slave-rwrg"
server_key: "a6d4329c95602175542270a52eebff6b"
server_key_iv: "3541f05fd31c8dbd5cf694fc9d945cfb"
server_key_version: "encryptionkey@0"
I20260807 06:34:10.756729 18071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:10.770874 18071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:10.771241 18071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:10.771353 18071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:10.775465 18071 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:45157
I20260807 06:34:10.775516 18176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:45157 every 8 connection(s)
I20260807 06:34:10.775889 18071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:34:10.776691 18177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:34:10.779199 18177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d: Bootstrap starting.
I20260807 06:34:10.779824 18177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d: Neither blocks nor log segments found. Creating new log.
I20260807 06:34:10.780071 18177 log.cc:824] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:34:10.780838 18177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d: No bootstrap required, opened a new log
I20260807 06:34:10.781719 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 18071
I20260807 06:34:10.781808 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:34:10.782009 3020 external_mini_cluster.cc:1428] Setting key 8cfe18b6bf4a0b5f7e085a8f04c1d541
I20260807 06:34:10.783375 18177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e45312631348d3979e93c53366012d" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45157 } }
I20260807 06:34:10.783577 18177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:34:10.783628 18177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e45312631348d3979e93c53366012d, State: Initialized, Role: FOLLOWER
I20260807 06:34:10.783787 18177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e45312631348d3979e93c53366012d" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45157 } }
I20260807 06:34:10.783882 18177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:34:10.783933 18177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:34:10.783995 18177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:34:10.784858 18177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e45312631348d3979e93c53366012d" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45157 } }
I20260807 06:34:10.784986 18177 leader_election.cc:302] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e45312631348d3979e93c53366012d; no voters:
I20260807 06:34:10.785156 18177 leader_election.cc:288] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:34:10.785228 18190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:34:10.785389 18190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [term 1 LEADER]: Becoming Leader. State: Replica: 01e45312631348d3979e93c53366012d, State: Running, Role: LEADER
I20260807 06:34:10.785486 18177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:34:10.785495 18190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e45312631348d3979e93c53366012d" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45157 } }
I20260807 06:34:10.785971 18190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e45312631348d3979e93c53366012d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e45312631348d3979e93c53366012d" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45157 } } }
I20260807 06:34:10.786059 18190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:10.786186 18190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e45312631348d3979e93c53366012d. Latest consensus state: current_term: 1 leader_uuid: "01e45312631348d3979e93c53366012d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e45312631348d3979e93c53366012d" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45157 } } }
I20260807 06:34:10.786252 18190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:10.786392 18203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:34:10.787582 18203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:34:10.792309 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:10.782710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54458 (local address 127.2.243.62:45157)
0807 06:34:10.783078 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:10.783087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:10.783123 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:34:10.783331 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:10.783338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:10.783432 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:10.783765 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:10.783949 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.786518 (+ 2569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.787719 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.788372 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.788540 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:10.791418 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:10.791423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:10.791423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:10.791625 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:10.791666 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:10.791678 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:10.791740 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:10.791986 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 06:34:10.792063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":138,"threads_started":1}
I20260807 06:34:10.795188 18203 catalog_manager.cc:1347] Generated new cluster ID: c19c9633351646818c9eca6c9375b9f3
I20260807 06:34:10.795248 18203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:34:10.805768 18203 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:34:10.806484 18203 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:34:10.812328 18203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e45312631348d3979e93c53366012d: Generated new TSK 0
I20260807 06:34:10.812541 18203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:34:10.818501 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45157
--builtin_ntp_servers=127.2.243.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 06:34:10.918001 18229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:10.918196 18229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:10.918217 18229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:10.919939 18229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:10.919999 18229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:10.920040 18229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:34:10.922250 18229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:10.922531 18229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:10.922919 18229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:10.930823 18234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:10.930863 18235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:10.931969 18238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:10.933084 18229 server_base.cc:1034] running on GCE node
I20260807 06:34:10.933329 18229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:10.933609 18229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:10.934760 18229 hybrid_clock.cc:648] HybridClock initialized: now 1786084450934746 us; error 27 us; skew 500 ppm
I20260807 06:34:10.936237 18229 webserver.cc:460] Webserver started at http://127.2.243.1:42997/ using document root <none> and password file <none>
I20260807 06:34:10.936513 18229 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:10.936587 18229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:10.936738 18229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:10.938064 18229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "f85281a045ec4c45b74808bc11bb1e29"
format_stamp: "Formatted at 2026-08-07 06:34:10 on dist-test-slave-rwrg"
server_key: "bdc5cf44183577386617277acb3dc308"
server_key_iv: "c1f2914422e1f8eca2d3a502b916909a"
server_key_version: "encryptionkey@0"
I20260807 06:34:10.938493 18229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "f85281a045ec4c45b74808bc11bb1e29"
format_stamp: "Formatted at 2026-08-07 06:34:10 on dist-test-slave-rwrg"
server_key: "bdc5cf44183577386617277acb3dc308"
server_key_iv: "c1f2914422e1f8eca2d3a502b916909a"
server_key_version: "encryptionkey@0"
I20260807 06:34:10.940199 18229 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 06:34:10.941150 18247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:10.941347 18229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:34:10.941473 18229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "f85281a045ec4c45b74808bc11bb1e29"
format_stamp: "Formatted at 2026-08-07 06:34:10 on dist-test-slave-rwrg"
server_key: "bdc5cf44183577386617277acb3dc308"
server_key_iv: "c1f2914422e1f8eca2d3a502b916909a"
server_key_version: "encryptionkey@0"
I20260807 06:34:10.941541 18229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:10.952450 18229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:10.954761 18229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:10.954923 18229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:10.955199 18229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:10.955616 18229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:10.955703 18229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:10.955806 18229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:10.955838 18229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:10.963557 18229 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:39815
I20260807 06:34:10.963652 18434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:39815 every 8 connection(s)
I20260807 06:34:10.964151 18229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:34:10.965075 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 18229
I20260807 06:34:10.965173 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:34:10.965445 3020 external_mini_cluster.cc:1428] Setting key 97efe56e321f5d124c3d0d50e117e922
I20260807 06:34:10.966635 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45157
--builtin_ntp_servers=127.2.243.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 06:34:10.977533 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:10.968571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:51515 (local address 127.2.243.62:45157)
0807 06:34:10.968675 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:10.968678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:10.968689 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:34:10.968717 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:10.968718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:10.968731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:10.968797 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:10.969796 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.970591 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.972306 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.972690 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.972767 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:10.977174 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:10.977175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:10.977175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:10.977181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:10.977200 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:10.977205 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:10.977236 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:10.977382 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 06:34:10.977417 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:34:10.978231 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:10.965017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45157 (local address 127.2.243.1:51515)
0807 06:34:10.965777 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:10.965843 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:10.966555 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:10.968912 (+ 2357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:10.968918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:10.968931 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:10.969701 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:10.969708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.970702 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.970703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:10.972198 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:10.972202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:10.972845 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:10.972847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:10.973069 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:10.976955 (+ 3886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:34:10.976956 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:10.977012 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:34:10.977047 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:10.977284 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:10.977290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:10.977294 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:34:10.977786 (+ 492us) client_negotiation.cc:240] Negotiation successful
0807 06:34:10.977937 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":108,"threads_started":1}
I20260807 06:34:10.978788 18436 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45157
I20260807 06:34:10.978890 18436 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:10.979112 18436 heartbeater.cc:511] Master 127.2.243.62:45157 requested a full tablet report, sending...
I20260807 06:34:10.979671 18120 ts_manager.cc:194] Registered new tserver with Master: f85281a045ec4c45b74808bc11bb1e29 (127.2.243.1:39815)
I20260807 06:34:10.980314 18120 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.1:51515
W20260807 06:34:11.051919 18445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:11.052124 18445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:11.052145 18445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:11.053309 18445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:11.053350 18445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:11.053375 18445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:34:11.054692 18445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:11.054920 18445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:11.055224 18445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:11.060576 18452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.060634 18450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.060637 18456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:11.061057 18445 server_base.cc:1034] running on GCE node
I20260807 06:34:11.061295 18445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:11.061573 18445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:11.062757 18445 hybrid_clock.cc:648] HybridClock initialized: now 1786084451062720 us; error 28 us; skew 500 ppm
I20260807 06:34:11.064095 18445 webserver.cc:460] Webserver started at http://127.2.243.2:38349/ using document root <none> and password file <none>
I20260807 06:34:11.064343 18445 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:11.064389 18445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:11.064608 18445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:11.065696 18445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "e8051b432323499b8e1e7f3a460d017f"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "1606c5c601dab209d1aa38f12289dcce"
server_key_iv: "62e1d6c049c2e92094619d60b03c3e2d"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.066066 18445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "e8051b432323499b8e1e7f3a460d017f"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "1606c5c601dab209d1aa38f12289dcce"
server_key_iv: "62e1d6c049c2e92094619d60b03c3e2d"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.067329 18445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:34:11.068114 18466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.068300 18445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:34:11.068392 18445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "e8051b432323499b8e1e7f3a460d017f"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "1606c5c601dab209d1aa38f12289dcce"
server_key_iv: "62e1d6c049c2e92094619d60b03c3e2d"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.068459 18445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:11.079360 18445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:11.079648 18445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:11.079798 18445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:11.080005 18445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:11.080340 18445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:11.080394 18445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.080433 18445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:11.080461 18445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.086580 18445 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:45905
I20260807 06:34:11.086656 18655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:45905 every 8 connection(s)
I20260807 06:34:11.087025 18445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:34:11.091949 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.087759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:60741 (local address 127.2.243.62:45157)
0807 06:34:11.087835 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.087837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.088094 (+ 257us) server_negotiation.cc:407] Connection header received
0807 06:34:11.088589 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.088590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.088599 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.088659 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.089285 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.089764 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.090786 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.090959 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.091013 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:11.091747 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:11.091749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:11.091749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:11.091754 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:11.091767 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:11.091771 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:11.091811 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:11.091872 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 06:34:11.091894 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:34:11.092168 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.087757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45157 (local address 127.2.243.2:60741)
0807 06:34:11.087953 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:11.087995 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:11.088467 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:11.088708 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:11.088711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:11.088719 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:11.089196 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:11.089199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.089870 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.089871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:11.090699 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:11.090702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.091036 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.091037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:11.091142 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:11.091617 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:34:11.091617 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:11.091644 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:34:11.091664 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:11.091812 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:11.091814 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:11.091816 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:34:11.091859 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 06:34:11.091931 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260807 06:34:11.092728 18656 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45157
I20260807 06:34:11.092830 18656 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:11.092998 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 18445
I20260807 06:34:11.093007 18656 heartbeater.cc:511] Master 127.2.243.62:45157 requested a full tablet report, sending...
I20260807 06:34:11.093079 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:34:11.093302 3020 external_mini_cluster.cc:1428] Setting key 3c2cefec2bf09823fb8012db08a3f6e4
I20260807 06:34:11.093467 18120 ts_manager.cc:194] Registered new tserver with Master: e8051b432323499b8e1e7f3a460d017f (127.2.243.2:45905)
I20260807 06:34:11.094165 18120 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.2:60741
I20260807 06:34:11.094563 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45157
--builtin_ntp_servers=127.2.243.20:38773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 06:34:11.178598 18659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:11.178834 18659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:11.178870 18659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:11.180033 18659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:11.180097 18659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:11.180143 18659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:34:11.181485 18659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:11.181802 18659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:11.182098 18659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:11.187321 18667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.187388 18670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.187527 18668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:11.187716 18659 server_base.cc:1034] running on GCE node
I20260807 06:34:11.187935 18659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:11.188208 18659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:11.189376 18659 hybrid_clock.cc:648] HybridClock initialized: now 1786084451189370 us; error 44 us; skew 500 ppm
I20260807 06:34:11.190603 18659 webserver.cc:460] Webserver started at http://127.2.243.3:36625/ using document root <none> and password file <none>
I20260807 06:34:11.190845 18659 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:11.190912 18659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:11.191048 18659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:11.192072 18659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "bb252b3555204e48a380a4b4636dfc48"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "5d25233b332aa257630b7fe0273f91a3"
server_key_iv: "f1c444ebb4728db21212984080949c4b"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.192425 18659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "bb252b3555204e48a380a4b4636dfc48"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "5d25233b332aa257630b7fe0273f91a3"
server_key_iv: "f1c444ebb4728db21212984080949c4b"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.193820 18659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:34:11.194523 18678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.194700 18659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:34:11.194798 18659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "bb252b3555204e48a380a4b4636dfc48"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "5d25233b332aa257630b7fe0273f91a3"
server_key_iv: "f1c444ebb4728db21212984080949c4b"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.194882 18659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:11.215783 18659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:11.216090 18659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:11.216236 18659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:11.216451 18659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:11.216801 18659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:11.216856 18659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.216887 18659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:11.216914 18659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.223088 18659 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:41621
I20260807 06:34:11.223132 18853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:41621 every 8 connection(s)
I20260807 06:34:11.223438 18659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:34:11.228935 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.224238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:39829 (local address 127.2.243.62:45157)
0807 06:34:11.224315 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.224318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.224687 (+ 369us) server_negotiation.cc:407] Connection header received
0807 06:34:11.225192 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.225193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.225204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.225257 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.225942 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.226416 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.227410 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.227628 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.227686 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:11.228460 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:11.228461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:11.228462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:11.228466 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:11.228480 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:11.228483 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:11.228713 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:11.228849 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 06:34:11.228876 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:34:11.229200 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.224209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45157 (local address 127.2.243.3:39829)
0807 06:34:11.224460 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:11.224534 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:11.225097 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:11.225327 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:11.225330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:11.225339 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:11.225868 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:11.225872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.226541 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.226542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:11.227306 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:11.227308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:11.227716 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:11.227717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:11.227825 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:11.228330 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:34:11.228331 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:11.228357 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:34:11.228377 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:11.228744 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:11.228751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:11.228755 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:34:11.228843 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 06:34:11.228924 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260807 06:34:11.229758 18854 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45157
I20260807 06:34:11.229857 18854 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:11.230016 18854 heartbeater.cc:511] Master 127.2.243.62:45157 requested a full tablet report, sending...
I20260807 06:34:11.230119 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 18659
I20260807 06:34:11.230227 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:34:11.230394 18120 ts_manager.cc:194] Registered new tserver with Master: bb252b3555204e48a380a4b4636dfc48 (127.2.243.3:41621)
I20260807 06:34:11.230428 3020 external_mini_cluster.cc:1428] Setting key 770f09111900887d492155ca0d15bb89
I20260807 06:34:11.230878 18120 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.3:39829
I20260807 06:34:11.231420 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:34:11.233347 18183 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:11.232830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.2.243.62:45157)
0807 06:34:11.232866 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.232868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.233019 (+ 151us) server_negotiation.cc:407] Connection header received
0807 06:34:11.233091 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.233092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.233103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.233159 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.233286 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54474: BlockingRecv error: recv got EOF from 127.0.0.1:54474 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:34:11.233441 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 18229
I20260807 06:34:11.248260 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 18445
I20260807 06:34:11.255231 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 18659
I20260807 06:34:11.264333 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 18071
2026-08-07T06:34:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (643 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T06:34:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:34:11Z Disabled control of system clock
I20260807 06:34:11.300688 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:35979
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:35979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:34:11.382964 18868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:11.383226 18868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:11.383263 18868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:11.384423 18868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:34:11.384495 18868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:11.384538 18868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:11.384572 18868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:34:11.384601 18868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:34:11.386076 18868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:35979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:35979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:11.386354 18868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:11.386655 18868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:11.391597 18879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.391615 18885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:11.391716 18868 server_base.cc:1034] running on GCE node
W20260807 06:34:11.391611 18878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:11.392038 18868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:11.392280 18868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:11.393416 18868 hybrid_clock.cc:648] HybridClock initialized: now 1786084451393362 us; error 81 us; skew 500 ppm
I20260807 06:34:11.394762 18868 webserver.cc:460] Webserver started at http://127.2.243.62:39319/ using document root <none> and password file <none>
I20260807 06:34:11.395099 18868 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:11.395156 18868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:11.395323 18868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:11.396437 18868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "51dfc5986ead48529a8cdbf640cb00ea"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "e2ae75e83b6d3cdcc2112ebfe6f2014a"
server_key_iv: "9ffaa6396731e0dd0d08d4705e72436f"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.396899 18868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "51dfc5986ead48529a8cdbf640cb00ea"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "e2ae75e83b6d3cdcc2112ebfe6f2014a"
server_key_iv: "9ffaa6396731e0dd0d08d4705e72436f"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.398429 18868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:34:11.399284 18894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.399471 18868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:34:11.399569 18868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "51dfc5986ead48529a8cdbf640cb00ea"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "e2ae75e83b6d3cdcc2112ebfe6f2014a"
server_key_iv: "9ffaa6396731e0dd0d08d4705e72436f"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.399631 18868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:11.402976 18868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:11.403199 18868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:11.403296 18868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:11.406955 18868 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:35979
I20260807 06:34:11.407024 18984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:35979 every 8 connection(s)
I20260807 06:34:11.407366 18868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:34:11.408034 18985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:34:11.410802 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea: Bootstrap starting.
I20260807 06:34:11.411435 18985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea: Neither blocks nor log segments found. Creating new log.
I20260807 06:34:11.411749 18985 log.cc:824] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea: Log is configured to *not* fsync() on all Append() calls
I20260807 06:34:11.412604 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea: No bootstrap required, opened a new log
I20260807 06:34:11.414202 18985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51dfc5986ead48529a8cdbf640cb00ea" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 35979 } }
I20260807 06:34:11.414359 18985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:34:11.414407 18985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51dfc5986ead48529a8cdbf640cb00ea, State: Initialized, Role: FOLLOWER
I20260807 06:34:11.414557 18985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51dfc5986ead48529a8cdbf640cb00ea" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 35979 } }
I20260807 06:34:11.414647 18985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:34:11.414690 18985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:34:11.414742 18985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:34:11.415294 18985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51dfc5986ead48529a8cdbf640cb00ea" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 35979 } }
I20260807 06:34:11.415359 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 18868
I20260807 06:34:11.415410 18985 leader_election.cc:302] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51dfc5986ead48529a8cdbf640cb00ea; no voters:
I20260807 06:34:11.415453 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:34:11.415618 18985 leader_election.cc:288] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:34:11.415649 18988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:34:11.415702 3020 external_mini_cluster.cc:1428] Setting key c8845fc2114716f6e83b0495ccd82b60
I20260807 06:34:11.415768 18988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [term 1 LEADER]: Becoming Leader. State: Replica: 51dfc5986ead48529a8cdbf640cb00ea, State: Running, Role: LEADER
I20260807 06:34:11.415853 18988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51dfc5986ead48529a8cdbf640cb00ea" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 35979 } }
I20260807 06:34:11.415894 18985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:34:11.416392 18988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51dfc5986ead48529a8cdbf640cb00ea. Latest consensus state: current_term: 1 leader_uuid: "51dfc5986ead48529a8cdbf640cb00ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51dfc5986ead48529a8cdbf640cb00ea" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 35979 } } }
I20260807 06:34:11.416484 18988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:11.416662 18989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51dfc5986ead48529a8cdbf640cb00ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51dfc5986ead48529a8cdbf640cb00ea" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 35979 } } }
I20260807 06:34:11.416740 18989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:11.416898 19003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:34:11.417438 19003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:34:11.419112 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.416238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35714 (local address 127.2.243.62:35979)
0807 06:34:11.417404 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.417414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.417448 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:34:11.417561 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.417569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.417613 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.417783 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.418569 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:11.418572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:11.418573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:11.418577 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:11.418629 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:11.418645 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:11.418726 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:11.418855 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 06:34:11.418918 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1083,"thread_start_us":86,"threads_started":1}
I20260807 06:34:11.419646 19003 catalog_manager.cc:1347] Generated new cluster ID: bce7c345170d4ba38e1eb209b205ea63
I20260807 06:34:11.419700 19003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:34:11.427838 19003 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:34:11.427999 19003 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:34:11.436301 19003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51dfc5986ead48529a8cdbf640cb00ea: Generated new TSK 0
I20260807 06:34:11.436483 19003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:34:11.443763 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:35979
--builtin_ntp_servers=127.2.243.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:34:11.528985 19019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:11.529201 19019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:11.529259 19019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:11.530423 19019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:11.530488 19019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:11.530532 19019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:34:11.531877 19019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:35979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:11.532124 19019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:11.532411 19019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:11.537478 19032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.537462 19031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.537566 19034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:11.537788 19019 server_base.cc:1034] running on GCE node
I20260807 06:34:11.538053 19019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:11.538380 19019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:11.539551 19019 hybrid_clock.cc:648] HybridClock initialized: now 1786084451539538 us; error 27 us; skew 500 ppm
I20260807 06:34:11.540940 19019 webserver.cc:460] Webserver started at http://127.2.243.1:37355/ using document root <none> and password file <none>
I20260807 06:34:11.541262 19019 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:11.541337 19019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:11.541471 19019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:11.542528 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "fa0b2bd63aa34a3489574f98c898bad7"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "1cdb2168ddd87d6c91a8e92e33555e62"
server_key_iv: "c0cc8afb54ae2a8e4edd5d7af4f672bf"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.542896 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "fa0b2bd63aa34a3489574f98c898bad7"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "1cdb2168ddd87d6c91a8e92e33555e62"
server_key_iv: "c0cc8afb54ae2a8e4edd5d7af4f672bf"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.544231 19019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:34:11.545056 19046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.545224 19019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.545351 19019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "fa0b2bd63aa34a3489574f98c898bad7"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "1cdb2168ddd87d6c91a8e92e33555e62"
server_key_iv: "c0cc8afb54ae2a8e4edd5d7af4f672bf"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.545404 19019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:11.548724 19019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:11.548949 19019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:11.549063 19019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:11.549258 19019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:11.549559 19019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:11.549593 19019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.549613 19019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:11.549626 19019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.555071 19019 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:45307
I20260807 06:34:11.555198 19229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:45307 every 8 connection(s)
I20260807 06:34:11.555467 19019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:34:11.558360 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.556191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:58571 (local address 127.2.243.62:35979)
0807 06:34:11.556263 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.556265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.556487 (+ 222us) server_negotiation.cc:407] Connection header received
0807 06:34:11.556987 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.556988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.556994 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.557056 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.558178 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:11.558179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:11.558179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:11.558182 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:11.558191 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:11.558195 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:11.558241 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:11.558299 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 06:34:11.558311 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:34:11.558564 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.556187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:35979 (local address 127.2.243.1:58571)
0807 06:34:11.556363 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:11.556399 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:11.556878 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:11.557103 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:11.557106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:11.557114 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:11.558053 (+ 939us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:34:11.558054 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:11.558080 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:34:11.558105 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:11.558240 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:11.558242 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:11.558244 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:34:11.558292 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 06:34:11.558335 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260807 06:34:11.559101 19230 heartbeater.cc:347] Connected to a master server at 127.2.243.62:35979
I20260807 06:34:11.559204 19230 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:11.559383 19230 heartbeater.cc:511] Master 127.2.243.62:35979 requested a full tablet report, sending...
I20260807 06:34:11.559567 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 19019
I20260807 06:34:11.559669 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:34:11.559859 3020 external_mini_cluster.cc:1428] Setting key 36f10b42f7f25746bb82c304197f7448
I20260807 06:34:11.559957 18920 ts_manager.cc:194] Registered new tserver with Master: fa0b2bd63aa34a3489574f98c898bad7 (127.2.243.1:45307)
I20260807 06:34:11.560923 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:35979
--builtin_ntp_servers=127.2.243.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:34:11.642396 19240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:11.642601 19240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:11.642624 19240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:11.643800 19240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:11.643846 19240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:11.643872 19240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:34:11.645215 19240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:35979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:11.645470 19240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:11.645709 19240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:11.650715 19250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.650780 19249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.650780 19253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:11.651147 19240 server_base.cc:1034] running on GCE node
I20260807 06:34:11.651325 19240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:11.651513 19240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:11.652658 19240 hybrid_clock.cc:648] HybridClock initialized: now 1786084451652645 us; error 34 us; skew 500 ppm
I20260807 06:34:11.653983 19240 webserver.cc:460] Webserver started at http://127.2.243.2:42653/ using document root <none> and password file <none>
I20260807 06:34:11.654234 19240 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:11.654282 19240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:11.654467 19240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:11.655639 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "4cee07e73c2648909f26b497c04a68ef"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "69430c7ca173619b2a4fc3ccb7cb64d0"
server_key_iv: "b6e80bc7498e2834621fded28f82f528"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.656011 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "4cee07e73c2648909f26b497c04a68ef"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "69430c7ca173619b2a4fc3ccb7cb64d0"
server_key_iv: "b6e80bc7498e2834621fded28f82f528"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.657470 19240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:34:11.658308 19265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.658548 19240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:34:11.658667 19240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "4cee07e73c2648909f26b497c04a68ef"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "69430c7ca173619b2a4fc3ccb7cb64d0"
server_key_iv: "b6e80bc7498e2834621fded28f82f528"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.658744 19240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:11.662191 19240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:11.662444 19240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:11.662593 19240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:11.662798 19240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:11.663090 19240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:11.663142 19240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.663180 19240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:11.663208 19240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.669152 19240 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:44799
I20260807 06:34:11.669196 19446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:44799 every 8 connection(s)
I20260807 06:34:11.669535 19240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:34:11.672653 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.670261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:52137 (local address 127.2.243.62:35979)
0807 06:34:11.670324 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.670327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.670604 (+ 277us) server_negotiation.cc:407] Connection header received
0807 06:34:11.671061 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.671062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.671067 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.671128 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.672373 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:11.672374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:11.672374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:11.672379 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:11.672394 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:11.672398 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:11.672449 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:11.672583 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 06:34:11.672601 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:34:11.672847 19449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.670222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:35979 (local address 127.2.243.2:52137)
0807 06:34:11.670426 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:11.670516 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:11.670972 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:11.671182 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:11.671185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:11.671192 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:11.672214 (+ 1022us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:34:11.672215 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:11.672244 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:34:11.672272 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:11.672448 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:11.672451 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:11.672453 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:34:11.672578 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 06:34:11.672639 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260807 06:34:11.673398 19447 heartbeater.cc:347] Connected to a master server at 127.2.243.62:35979
I20260807 06:34:11.673503 19447 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:11.673624 19447 heartbeater.cc:511] Master 127.2.243.62:35979 requested a full tablet report, sending...
I20260807 06:34:11.674023 18919 ts_manager.cc:194] Registered new tserver with Master: 4cee07e73c2648909f26b497c04a68ef (127.2.243.2:44799)
I20260807 06:34:11.675922 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 19240
I20260807 06:34:11.676016 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:34:11.676213 3020 external_mini_cluster.cc:1428] Setting key 436926568b594bb10065e9e69de14efa
I20260807 06:34:11.677196 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:35979
--builtin_ntp_servers=127.2.243.20:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:34:11.762861 19451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:11.763088 19451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:11.763125 19451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:11.764273 19451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:11.764343 19451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:11.764386 19451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:34:11.765884 19451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:35979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:11.766191 19451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:11.766501 19451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:11.771729 19468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.771729 19464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:11.771782 19462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:11.771814 19451 server_base.cc:1034] running on GCE node
I20260807 06:34:11.772136 19451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:11.772378 19451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:11.773569 19451 hybrid_clock.cc:648] HybridClock initialized: now 1786084451773569 us; error 41 us; skew 500 ppm
I20260807 06:34:11.774895 19451 webserver.cc:460] Webserver started at http://127.2.243.3:33709/ using document root <none> and password file <none>
I20260807 06:34:11.775154 19451 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:11.775202 19451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:11.775357 19451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:11.776482 19451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "e6163ec6e3794e918bb4d7f35279d123"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "a8edea0472a89dd4508dadaf0fab3ae9"
server_key_iv: "2f316e38834c6a8c43757520afa59034"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.776918 19451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "e6163ec6e3794e918bb4d7f35279d123"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "a8edea0472a89dd4508dadaf0fab3ae9"
server_key_iv: "2f316e38834c6a8c43757520afa59034"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.778265 19451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:34:11.779067 19484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.779273 19451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:34:11.779364 19451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "e6163ec6e3794e918bb4d7f35279d123"
format_stamp: "Formatted at 2026-08-07 06:34:11 on dist-test-slave-rwrg"
server_key: "a8edea0472a89dd4508dadaf0fab3ae9"
server_key_iv: "2f316e38834c6a8c43757520afa59034"
server_key_version: "encryptionkey@0"
I20260807 06:34:11.779438 19451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:11.782970 19451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:11.783229 19451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:11.783365 19451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:11.783571 19451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:11.783883 19451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:11.783946 19451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.783978 19451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:11.784004 19451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:11.790027 19451 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:33367
I20260807 06:34:11.790082 19678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:33367 every 8 connection(s)
I20260807 06:34:11.790387 19451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:34:11.792759 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 19451
I20260807 06:34:11.792883 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:34:11.793115 3020 external_mini_cluster.cc:1428] Setting key 82c7c02e5882b7fe7aa78785258110c3
I20260807 06:34:11.793879 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.791110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:57151 (local address 127.2.243.62:35979)
0807 06:34:11.791155 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.791157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.791426 (+ 269us) server_negotiation.cc:407] Connection header received
0807 06:34:11.791964 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.791966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.791973 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.792029 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.793587 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:11.793588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:11.793589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:34:11.793594 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:11.793612 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:34:11.793621 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:11.793693 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:11.793791 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 06:34:11.793807 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:34:11.794097 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:11.791064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:35979 (local address 127.2.243.3:57151)
0807 06:34:11.791300 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:11.791335 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:11.791841 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:11.792102 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:11.792106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:11.792116 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:11.793387 (+ 1271us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:34:11.793388 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:11.793417 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:34:11.793445 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:11.793691 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:11.793698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:11.793701 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:34:11.793801 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:34:11.793864 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":108,"threads_started":1}
I20260807 06:34:11.794528 19681 heartbeater.cc:347] Connected to a master server at 127.2.243.62:35979
I20260807 06:34:11.794602 19681 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:11.794764 19681 heartbeater.cc:511] Master 127.2.243.62:35979 requested a full tablet report, sending...
I20260807 06:34:11.795121 18919 ts_manager.cc:194] Registered new tserver with Master: e6163ec6e3794e918bb4d7f35279d123 (127.2.243.3:33367)
I20260807 06:34:11.805111 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:34:11.807058 18994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:34:11.806598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35730 (local address 127.2.243.62:35979)
0807 06:34:11.806641 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:11.806643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:11.806733 (+ 90us) server_negotiation.cc:407] Connection header received
0807 06:34:11.806768 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:11.806769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:11.806772 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:11.806823 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:11.806974 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35730: BlockingRecv error: recv got EOF from 127.0.0.1:35730 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:34:11.807123 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 19019
I20260807 06:34:11.820176 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 19240
I20260807 06:34:11.838937 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 19451
I20260807 06:34:11.845909 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 18868
2026-08-07T06:34:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (579 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:34:11 dist-test-slave-rwrg krb5kdc[19696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:34:11 dist-test-slave-rwrg krb5kdc[19696](info): set up 2 sockets
Aug 07 06:34:11 dist-test-slave-rwrg krb5kdc[19696](info): commencing operation
krb5kdc: starting...
W20260807 06:34:13.110481 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:34:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:34:13.231294 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33401
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:33401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:34:13.312157 19717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:13.312388 19717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:13.312430 19717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:13.313645 19717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:34:13.313711 19717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:13.313735 19717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:13.313755 19717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:34:13.313802 19717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:34:13.315305 19717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:33401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:13.315584 19717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:13.315821 19717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:13.321069 19734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.321043 19725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.321064 19731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:13.321267 19717 server_base.cc:1034] running on GCE node
I20260807 06:34:13.321504 19717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:13.321799 19717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:13.322942 19717 hybrid_clock.cc:648] HybridClock initialized: now 1786084453322918 us; error 47 us; skew 500 ppm
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:34:13.325465 19717 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:34:13.325976 19717 webserver.cc:460] Webserver started at http://127.2.243.62:46067/ using document root <none> and password file <none>
I20260807 06:34:13.326241 19717 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:13.326311 19717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:13.326480 19717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:13.327502 19717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "1f7db2e2cf424388bc4663f0ce520f5f"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "bf4a7aadcf2a2c9c8f2d69d281e2f7df"
server_key_iv: "9d54c57f43324244c5f34e9b9e1ad525"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.327910 19717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "1f7db2e2cf424388bc4663f0ce520f5f"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "bf4a7aadcf2a2c9c8f2d69d281e2f7df"
server_key_iv: "9d54c57f43324244c5f34e9b9e1ad525"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.329380 19717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:34:13.330217 19745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.330411 19717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:34:13.330525 19717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "1f7db2e2cf424388bc4663f0ce520f5f"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "bf4a7aadcf2a2c9c8f2d69d281e2f7df"
server_key_iv: "9d54c57f43324244c5f34e9b9e1ad525"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.330622 19717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:13.351852 19717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:13.353875 19717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:13.354040 19717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:13.358191 19717 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:33401
I20260807 06:34:13.358235 19816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:33401 every 8 connection(s)
I20260807 06:34:13.358633 19717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:34:13.359640 19817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:34:13.362105 19817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f: Bootstrap starting.
I20260807 06:34:13.362630 19817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f: Neither blocks nor log segments found. Creating new log.
I20260807 06:34:13.362857 19817 log.cc:824] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:34:13.363608 19817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f: No bootstrap required, opened a new log
I20260807 06:34:13.365159 19817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33401 } }
I20260807 06:34:13.365290 19817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:34:13.365314 19817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7db2e2cf424388bc4663f0ce520f5f, State: Initialized, Role: FOLLOWER
I20260807 06:34:13.365461 19817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33401 } }
I20260807 06:34:13.365551 19817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:34:13.365576 19817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:34:13.365609 19817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:34:13.366338 19817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33401 } }
I20260807 06:34:13.366425 19817 leader_election.cc:302] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f7db2e2cf424388bc4663f0ce520f5f; no voters:
I20260807 06:34:13.366457 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 19717
I20260807 06:34:13.366634 19817 leader_election.cc:288] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:34:13.366631 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:34:13.366694 19822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:34:13.366876 3020 external_mini_cluster.cc:1428] Setting key 95605087e50006b6a50743f8abc8ddf5
I20260807 06:34:13.366876 19822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [term 1 LEADER]: Becoming Leader. State: Replica: 1f7db2e2cf424388bc4663f0ce520f5f, State: Running, Role: LEADER
I20260807 06:34:13.366910 19817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:34:13.366981 19822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33401 } }
I20260807 06:34:13.367460 19824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33401 } } }
I20260807 06:34:13.367551 19824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:13.367765 19825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f7db2e2cf424388bc4663f0ce520f5f. Latest consensus state: current_term: 1 leader_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7db2e2cf424388bc4663f0ce520f5f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33401 } } }
I20260807 06:34:13.367815 19836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:34:13.367902 19825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:13.368436 19836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:34:13.370428 19836 catalog_manager.cc:1347] Generated new cluster ID: ed7f88f82dce4aada5c11bed1bb334f3
I20260807 06:34:13.370486 19836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:34:13.375655 19836 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:34:13.376410 19836 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:34:13.379964 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.367640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.2.243.62:33401)
0807 06:34:13.368311 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.368341 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.368373 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:34:13.368730 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.368848 (+ 118us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.368905 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.369073 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:13.369923 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.371203 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.372174 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.372816 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.372964 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.375883 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:13.375887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:13.375887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:34:13.375899 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:13.378041 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.378501 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.378505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.378505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.378616 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.378870 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.378872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.378872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.379137 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:13.379344 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.379485 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.379669 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":93,"threads_started":1}
I20260807 06:34:13.380110 19836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f: Generated new TSK 0
I20260807 06:34:13.380270 19836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:34:13.411671 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33401
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:34:13.495942 19856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:13.496161 19856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:13.496228 19856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:13.497432 19856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:13.497476 19856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:13.497503 19856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:34:13.498963 19856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:13.499199 19856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:13.499436 19856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:13.505340 19866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.505353 19864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.505362 19869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:13.505678 19856 server_base.cc:1034] running on GCE node
I20260807 06:34:13.506070 19856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:13.506382 19856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:13.507544 19856 hybrid_clock.cc:648] HybridClock initialized: now 1786084453507531 us; error 32 us; skew 500 ppm
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:34:13.510255 19856 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:34:13.510749 19856 webserver.cc:460] Webserver started at http://127.2.243.1:33401/ using document root <none> and password file <none>
I20260807 06:34:13.510977 19856 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:13.511045 19856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:13.511194 19856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:13.512226 19856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "cc311562db914f44b5c26d48ab2eb63d"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "a7b602f212ae648b964d9f38ccb7f35f"
server_key_iv: "7c9eec53c56048044522eea048088515"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.512652 19856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "cc311562db914f44b5c26d48ab2eb63d"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "a7b602f212ae648b964d9f38ccb7f35f"
server_key_iv: "7c9eec53c56048044522eea048088515"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.514008 19856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:34:13.514725 19881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.514901 19856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:34:13.515000 19856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "cc311562db914f44b5c26d48ab2eb63d"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "a7b602f212ae648b964d9f38ccb7f35f"
server_key_iv: "7c9eec53c56048044522eea048088515"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.515086 19856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:13.537695 19856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:13.540076 19856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:13.540186 19856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:13.540388 19856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:13.540730 19856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:13.540763 19856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.540786 19856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:13.540797 19856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.546659 19856 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:33659
I20260807 06:34:13.546767 20069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:33659 every 8 connection(s)
I20260807 06:34:13.547081 19856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:34:13.556056 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.547837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:58669 (local address 127.2.243.62:33401)
0807 06:34:13.547893 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.547895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.548098 (+ 203us) server_negotiation.cc:407] Connection header received
0807 06:34:13.548620 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.548621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.548632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.548694 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:13.549646 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.550152 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.551160 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.551362 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.551412 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.553600 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:13.553602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:13.553602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:34:13.553608 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:13.554818 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.555236 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.555237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.555237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.555287 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.555533 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.555534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.555534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.555631 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:13.555679 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.555925 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.555991 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:34:13.556269 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.547789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33401 (local address 127.2.243.1:58669)
0807 06:34:13.547953 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:13.547996 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:13.548478 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:13.548762 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:13.548765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:13.549366 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:13.549563 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.549568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.550251 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.550252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.551080 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.551083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.551436 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.551436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.551560 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.552046 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:13.552055 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:13.553513 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:13.554910 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:34:13.554918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:34:13.554920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:34:13.555163 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:34:13.555352 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:34:13.555353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:34:13.555353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:34:13.555464 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:34:13.555690 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:13.555695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:13.555812 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 06:34:13.555901 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:34:13.556041 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260807 06:34:13.556919 20071 heartbeater.cc:347] Connected to a master server at 127.2.243.62:33401
I20260807 06:34:13.557039 20071 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:13.557224 20071 heartbeater.cc:511] Master 127.2.243.62:33401 requested a full tablet report, sending...
I20260807 06:34:13.557229 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 19856
I20260807 06:34:13.557364 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:34:13.557607 3020 external_mini_cluster.cc:1428] Setting key 8d9c28d838844ea1bc67b512e69dd975
I20260807 06:34:13.557897 19767 ts_manager.cc:194] Registered new tserver with Master: cc311562db914f44b5c26d48ab2eb63d (127.2.243.1:33659)
I20260807 06:34:13.558660 19767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:58669
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:34:13.584244 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33401
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:34:13.667970 20083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:13.668211 20083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:13.668236 20083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:13.669498 20083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:13.669543 20083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:13.669574 20083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:34:13.670948 20083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:13.671126 20083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:13.671291 20083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:13.676249 20100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.676280 20099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.676280 20103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:13.676368 20083 server_base.cc:1034] running on GCE node
I20260807 06:34:13.676646 20083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:13.676836 20083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:13.677973 20083 hybrid_clock.cc:648] HybridClock initialized: now 1786084453677947 us; error 40 us; skew 500 ppm
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:34:13.680405 20083 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:34:13.680900 20083 webserver.cc:460] Webserver started at http://127.2.243.2:36921/ using document root <none> and password file <none>
I20260807 06:34:13.681110 20083 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:13.681152 20083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:13.681237 20083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:13.682240 20083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "ace195ad27424977b51cf90d312698bf"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "5014ddef17ac60adc5e4858a9cec4f0f"
server_key_iv: "6748a2d42c28f0317a67f186394d4ac8"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.682535 20083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "ace195ad27424977b51cf90d312698bf"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "5014ddef17ac60adc5e4858a9cec4f0f"
server_key_iv: "6748a2d42c28f0317a67f186394d4ac8"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.683771 20083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:34:13.684455 20113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.684648 20083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:34:13.684741 20083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "ace195ad27424977b51cf90d312698bf"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "5014ddef17ac60adc5e4858a9cec4f0f"
server_key_iv: "6748a2d42c28f0317a67f186394d4ac8"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.684823 20083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:13.701069 20083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:13.703014 20083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:13.703188 20083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:13.703439 20083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:13.703773 20083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:13.703830 20083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.703958 20083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:13.704049 20083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.712255 20083 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:36539
I20260807 06:34:13.712327 20289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:36539 every 8 connection(s)
I20260807 06:34:13.712760 20083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:34:13.719888 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 20083
I20260807 06:34:13.720021 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:34:13.720276 3020 external_mini_cluster.cc:1428] Setting key 7a3ef7c53d864a87efceafa0b6c66525
I20260807 06:34:13.724104 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.713747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:43191 (local address 127.2.243.62:33401)
0807 06:34:13.713983 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.713985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.714093 (+ 108us) server_negotiation.cc:407] Connection header received
0807 06:34:13.714864 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.714865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.714882 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.714963 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:13.716339 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.716982 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.718117 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.718309 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.718389 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.720751 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:13.720752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:13.720752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:34:13.720759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:13.722538 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.723019 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.723020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.723021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.723084 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.723349 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.723350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.723350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.723502 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:13.723568 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.723908 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.724012 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260807 06:34:13.724295 20294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.713753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33401 (local address 127.2.243.2:43191)
0807 06:34:13.713937 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:13.713990 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:13.714724 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:13.715125 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:13.715142 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:13.716008 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:13.716217 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.716223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.717070 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.717071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.718036 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.718038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.718402 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.718403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.718499 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.719033 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:13.719042 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:13.720645 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:13.722635 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:34:13.722641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:34:13.722643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:34:13.722925 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:34:13.723163 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:34:13.723164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:34:13.723164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:34:13.723274 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:34:13.723560 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:13.723566 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:13.723714 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 06:34:13.723882 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 06:34:13.724059 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260807 06:34:13.724856 20291 heartbeater.cc:347] Connected to a master server at 127.2.243.62:33401
I20260807 06:34:13.724983 20291 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:13.725196 20291 heartbeater.cc:511] Master 127.2.243.62:33401 requested a full tablet report, sending...
I20260807 06:34:13.725593 19767 ts_manager.cc:194] Registered new tserver with Master: ace195ad27424977b51cf90d312698bf (127.2.243.2:36539)
I20260807 06:34:13.725956 19767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:43191
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:34:13.751232 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33401
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:34:13.834921 20298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:13.835144 20298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:13.835206 20298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:13.836423 20298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:13.836467 20298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:13.836535 20298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:34:13.838086 20298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:13.838361 20298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:13.838600 20298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:13.843923 20306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.843941 20305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:13.843941 20308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:13.844151 20298 server_base.cc:1034] running on GCE node
I20260807 06:34:13.844400 20298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:13.844738 20298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:13.845887 20298 hybrid_clock.cc:648] HybridClock initialized: now 1786084453845881 us; error 37 us; skew 500 ppm
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:34:13.848618 20298 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:34:13.849036 20298 webserver.cc:460] Webserver started at http://127.2.243.3:34723/ using document root <none> and password file <none>
I20260807 06:34:13.849264 20298 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:13.849310 20298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:13.849407 20298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:34:13.850489 20298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "219e9688539776f3761d56ae864ec96e"
server_key_iv: "6c594abb0c8c10f6a25c92e30c88bb2b"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.850813 20298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "219e9688539776f3761d56ae864ec96e"
server_key_iv: "6c594abb0c8c10f6a25c92e30c88bb2b"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.852236 20298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:34:13.853056 20321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.853225 20298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.853322 20298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2"
format_stamp: "Formatted at 2026-08-07 06:34:13 on dist-test-slave-rwrg"
server_key: "219e9688539776f3761d56ae864ec96e"
server_key_iv: "6c594abb0c8c10f6a25c92e30c88bb2b"
server_key_version: "encryptionkey@0"
I20260807 06:34:13.853413 20298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:13.877732 20298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:13.879951 20298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:13.880141 20298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:13.880421 20298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:34:13.880808 20298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:34:13.880846 20298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.880870 20298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:34:13.880884 20298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:13.887399 20298 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:39225
I20260807 06:34:13.887475 20503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:39225 every 8 connection(s)
I20260807 06:34:13.887828 20298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:34:13 dist-test-slave-rwrg krb5kdc[19696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084453, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:34:13.896752 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 20298
I20260807 06:34:13.896833 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:34:13.897060 3020 external_mini_cluster.cc:1428] Setting key 0bb4bca279bd5cd95c377c84ac64e344
I20260807 06:34:13.898706 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.888755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:38151 (local address 127.2.243.62:33401)
0807 06:34:13.888804 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.888806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.889032 (+ 226us) server_negotiation.cc:407] Connection header received
0807 06:34:13.889600 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.889602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.889615 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.889682 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:13.890700 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.891198 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.892290 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.892467 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.892597 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.895223 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:13.895225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:13.895225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:34:13.895232 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:13.896950 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.897491 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.897492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.897493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.897591 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.897936 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.897942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.897942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.898125 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:13.898207 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.898489 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.898617 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:34:13.898847 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.888738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33401 (local address 127.2.243.3:38151)
0807 06:34:13.888898 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:13.888937 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:13.889457 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:13.889738 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:13.889741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:13.890371 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:34:13.890582 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.890586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.891291 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.891292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.892210 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.892212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.892602 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.892603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.892727 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.893218 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:34:13.893227 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:34:13.895125 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:34:13.897061 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:34:13.897069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:34:13.897072 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:34:13.897405 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:34:13.897681 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:34:13.897682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:34:13.897683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:34:13.897838 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:34:13.898197 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:34:13.898204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:34:13.898350 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 06:34:13.898455 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:34:13.898620 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260807 06:34:13.899353 20504 heartbeater.cc:347] Connected to a master server at 127.2.243.62:33401
I20260807 06:34:13.899457 20504 heartbeater.cc:464] Registering TS with master...
I20260807 06:34:13.899614 20504 heartbeater.cc:511] Master 127.2.243.62:33401 requested a full tablet report, sending...
I20260807 06:34:13.899981 19767 ts_manager.cc:194] Registered new tserver with Master: 81d3ef0ec7f948c0bbfa713f36d3a7d2 (127.2.243.3:39225)
I20260807 06:34:13.900617 19767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:38151
I20260807 06:34:13.909227 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:34:13.917119 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.910617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.2.243.62:33401)
0807 06:34:13.910680 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.910682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.910844 (+ 162us) server_negotiation.cc:407] Connection header received
0807 06:34:13.910890 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.910891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.910901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.910948 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:13.911548 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.912123 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.912944 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.913130 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.913182 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.914423 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:13.914425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:13.914425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:34:13.914432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:13.915847 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.916180 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.916181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.916181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.916234 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:13.916439 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:13.916440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:13.916441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:13.916730 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:13.916827 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.916930 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.917006 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:34:13.918879 19767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:34:13.920599 19767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:34:13.927758 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.923259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:39225 (local address 127.0.0.1:44952)
0807 06:34:13.923621 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:13.923668 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:13.923740 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:13.924035 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:13.924050 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:13.924052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:34:13.924121 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.924123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.925497 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.925499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.926575 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.926578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.927458 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.927459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.927528 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.927539 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 06:34:13.927616 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:34:13.927658 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":38,"thread_start_us":62,"threads_started":1}
I20260807 06:34:13.927790 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.923369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44952 (local address 127.2.243.3:39225)
0807 06:34:13.923691 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.923696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.923712 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:34:13.923829 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.923831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.923848 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.923939 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:34:13.924209 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.925380 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.926665 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.927363 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.927443 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.927502 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.927652 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.927701 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":88,"threads_started":1}
I20260807 06:34:13.928164 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.923326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:33659 (local address 127.0.0.1:34038)
0807 06:34:13.923898 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:13.923908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:13.923961 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:13.924274 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:13.924275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:13.924281 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:34:13.924325 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.924326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.925601 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.925601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.926839 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.926841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.927711 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.927712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.927770 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.927775 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:34:13.927831 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:34:13.928096 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":8,"thread_start_us":102,"threads_started":1}
I20260807 06:34:13.928664 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.923456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.2.243.1:33659)
0807 06:34:13.923661 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.923667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.924038 (+ 371us) server_negotiation.cc:407] Connection header received
0807 06:34:13.924088 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.924091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.924107 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.924186 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:34:13.924386 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.925490 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.926919 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.927621 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.927694 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.927748 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.928535 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.928589 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260807 06:34:13.928582 20396 tablet_service.cc:1467] Processing CreateTablet for tablet 6da5ccbcd94e453194e61d86baec7492 (DEFAULT_TABLE table=test-table [id=fc5a435bb34943d49687c2d391a1b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:34:13.928969 20396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da5ccbcd94e453194e61d86baec7492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:34:13.929142 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.923466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:36539 (local address 127.0.0.1:51688)
0807 06:34:13.924028 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:13.924037 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:13.924253 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:13.924642 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:13.924643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:13.924644 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:34:13.924671 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.924672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.925562 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.925563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.926826 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:13.926829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.928896 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.928897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:13.928952 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.928955 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:34:13.929031 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:34:13.929074 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":116,"thread_start_us":95,"threads_started":1}
I20260807 06:34:13.929278 19969 tablet_service.cc:1467] Processing CreateTablet for tablet 6da5ccbcd94e453194e61d86baec7492 (DEFAULT_TABLE table=test-table [id=fc5a435bb34943d49687c2d391a1b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:34:13.929410 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:13.923715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.2.243.2:36539)
0807 06:34:13.924412 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:13.924415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:13.924423 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:34:13.924453 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:13.924455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:13.924466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:13.924579 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:34:13.924743 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.925435 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.927788 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:13.928729 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:13.928806 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:13.928860 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:13.929101 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 06:34:13.929139 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
I20260807 06:34:13.929615 19969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da5ccbcd94e453194e61d86baec7492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:34:13.932476 20534 tablet_bootstrap.cc:492] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Bootstrap starting.
I20260807 06:34:13.932847 20535 tablet_bootstrap.cc:492] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: Bootstrap starting.
I20260807 06:34:13.933235 20534 tablet_bootstrap.cc:654] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Neither blocks nor log segments found. Creating new log.
I20260807 06:34:13.933539 20535 tablet_bootstrap.cc:654] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: Neither blocks nor log segments found. Creating new log.
I20260807 06:34:13.933429 20188 tablet_service.cc:1467] Processing CreateTablet for tablet 6da5ccbcd94e453194e61d86baec7492 (DEFAULT_TABLE table=test-table [id=fc5a435bb34943d49687c2d391a1b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:34:13.933790 20188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da5ccbcd94e453194e61d86baec7492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:34:13.933879 20534 log.cc:824] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:34:13.934141 20535 log.cc:824] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:34:13.934821 20534 tablet_bootstrap.cc:492] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: No bootstrap required, opened a new log
I20260807 06:34:13.934887 20534 ts_tablet_manager.cc:1397] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:34:13.936225 20534 raft_consensus.cc:348] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:13.936383 20534 raft_consensus.cc:374] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:34:13.936429 20534 raft_consensus.cc:729] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d3ef0ec7f948c0bbfa713f36d3a7d2, State: Initialized, Role: FOLLOWER
I20260807 06:34:13.936478 20544 tablet_bootstrap.cc:492] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Bootstrap starting.
I20260807 06:34:13.936590 20534 consensus_queue.cc:260] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:13.936933 20534 ts_tablet_manager.cc:1428] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:34:13.936969 20504 heartbeater.cc:503] Master 127.2.243.62:33401 was elected leader, sending a full tablet report...
I20260807 06:34:13.937167 20544 tablet_bootstrap.cc:654] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Neither blocks nor log segments found. Creating new log.
I20260807 06:34:13.937609 20535 tablet_bootstrap.cc:492] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: No bootstrap required, opened a new log
I20260807 06:34:13.937687 20535 ts_tablet_manager.cc:1397] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 06:34:13.937714 20544 log.cc:824] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Log is configured to *not* fsync() on all Append() calls
I20260807 06:34:13.938506 20544 tablet_bootstrap.cc:492] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: No bootstrap required, opened a new log
I20260807 06:34:13.938594 20544 ts_tablet_manager.cc:1397] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:34:13.939390 20535 raft_consensus.cc:348] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:13.939512 20535 raft_consensus.cc:374] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:34:13.939533 20535 raft_consensus.cc:729] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc311562db914f44b5c26d48ab2eb63d, State: Initialized, Role: FOLLOWER
I20260807 06:34:13.939661 20535 consensus_queue.cc:260] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:13.939953 20535 ts_tablet_manager.cc:1428] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:34:13.939953 20071 heartbeater.cc:503] Master 127.2.243.62:33401 was elected leader, sending a full tablet report...
I20260807 06:34:13.940164 20544 raft_consensus.cc:348] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:13.940318 20544 raft_consensus.cc:374] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:34:13.940371 20544 raft_consensus.cc:729] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace195ad27424977b51cf90d312698bf, State: Initialized, Role: FOLLOWER
I20260807 06:34:13.940482 20544 consensus_queue.cc:260] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:13.940805 20544 ts_tablet_manager.cc:1428] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:34:13.940821 20291 heartbeater.cc:503] Master 127.2.243.62:33401 was elected leader, sending a full tablet report...
W20260807 06:34:13.963380 20293 tablet.cc:2369] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:34:14.047878 20073 tablet.cc:2369] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:34:14.063280 20548 raft_consensus.cc:482] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:34:14.063413 20548 raft_consensus.cc:504] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:14.063832 20548 leader_election.cc:288] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81d3ef0ec7f948c0bbfa713f36d3a7d2 (127.2.243.3:39225), ace195ad27424977b51cf90d312698bf (127.2.243.2:36539)
I20260807 06:34:14.067385 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:14.064201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:32873 (local address 127.2.243.3:39225)
0807 06:34:14.064285 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:14.064288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:14.064412 (+ 124us) server_negotiation.cc:407] Connection header received
0807 06:34:14.064480 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:14.064481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:14.064528 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:14.064601 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:34:14.064812 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.065349 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.066493 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.066983 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.067049 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:14.067059 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:14.067243 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 06:34:14.067280 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:34:14.067448 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:14.064080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:39225 (local address 127.2.243.1:32873)
0807 06:34:14.064300 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:14.064313 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:14.064407 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:14.064623 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:14.064624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:14.064626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:34:14.064728 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:14.064731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.065434 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.065435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:14.066394 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:14.066397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.067068 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.067069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:14.067148 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:14.067153 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:34:14.067224 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:34:14.067270 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":34,"thread_start_us":132,"threads_started":1}
I20260807 06:34:14.067655 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:14.064458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:44185 (local address 127.2.243.2:36539)
0807 06:34:14.064591 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:14.064593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:14.064605 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:34:14.064680 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:14.064681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:14.064696 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:14.064765 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:34:14.064974 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.065573 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.066568 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.067154 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.067233 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:14.067244 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:14.067460 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 06:34:14.067568 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:34:14.067904 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:14.064335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:36539 (local address 127.2.243.1:44185)
0807 06:34:14.064536 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:14.064547 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:34:14.064604 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:34:14.064795 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:34:14.064796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:34:14.064799 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:34:14.064888 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:14.064891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.065673 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.065674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:14.066458 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:34:14.066461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.067249 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.067251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:34:14.067345 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:14.067349 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:34:14.067743 (+ 394us) client_negotiation.cc:240] Negotiation successful
0807 06:34:14.067789 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260807 06:34:14.067988 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da5ccbcd94e453194e61d86baec7492" candidate_uuid: "cc311562db914f44b5c26d48ab2eb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" is_pre_election: true
I20260807 06:34:14.068185 20438 raft_consensus.cc:2454] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc311562db914f44b5c26d48ab2eb63d in term 0.
I20260807 06:34:14.068285 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da5ccbcd94e453194e61d86baec7492" candidate_uuid: "cc311562db914f44b5c26d48ab2eb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace195ad27424977b51cf90d312698bf" is_pre_election: true
I20260807 06:34:14.068472 20220 raft_consensus.cc:2454] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc311562db914f44b5c26d48ab2eb63d in term 0.
I20260807 06:34:14.068459 19885 leader_election.cc:302] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81d3ef0ec7f948c0bbfa713f36d3a7d2, cc311562db914f44b5c26d48ab2eb63d; no voters:
I20260807 06:34:14.068621 20548 raft_consensus.cc:2781] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:34:14.068714 20548 raft_consensus.cc:482] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:34:14.068755 20548 raft_consensus.cc:3037] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:34:14.069622 20548 raft_consensus.cc:504] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:14.069784 20548 leader_election.cc:288] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [CANDIDATE]: Term 1 election: Requested vote from peers 81d3ef0ec7f948c0bbfa713f36d3a7d2 (127.2.243.3:39225), ace195ad27424977b51cf90d312698bf (127.2.243.2:36539)
I20260807 06:34:14.069981 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da5ccbcd94e453194e61d86baec7492" candidate_uuid: "cc311562db914f44b5c26d48ab2eb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2"
I20260807 06:34:14.069999 20220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da5ccbcd94e453194e61d86baec7492" candidate_uuid: "cc311562db914f44b5c26d48ab2eb63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace195ad27424977b51cf90d312698bf"
I20260807 06:34:14.070084 20438 raft_consensus.cc:3037] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:34:14.070079 20220 raft_consensus.cc:3037] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:34:14.070839 20438 raft_consensus.cc:2454] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc311562db914f44b5c26d48ab2eb63d in term 1.
I20260807 06:34:14.070901 20220 raft_consensus.cc:2454] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc311562db914f44b5c26d48ab2eb63d in term 1.
I20260807 06:34:14.071048 19885 leader_election.cc:302] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81d3ef0ec7f948c0bbfa713f36d3a7d2, cc311562db914f44b5c26d48ab2eb63d; no voters:
I20260807 06:34:14.071161 20548 raft_consensus.cc:2781] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:34:14.071362 20548 raft_consensus.cc:686] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 1 LEADER]: Becoming Leader. State: Replica: cc311562db914f44b5c26d48ab2eb63d, State: Running, Role: LEADER
I20260807 06:34:14.071480 20548 consensus_queue.cc:237] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } }
I20260807 06:34:14.072427 19767 catalog_manager.cc:5495] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d reported cstate change: term changed from 0 to 1, leader changed from <none> to cc311562db914f44b5c26d48ab2eb63d (127.2.243.1). New cstate: current_term: 1 leader_uuid: "cc311562db914f44b5c26d48ab2eb63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc311562db914f44b5c26d48ab2eb63d" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33659 } health_report { overall_health: HEALTHY } } }
I20260807 06:34:14.108577 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:14.106169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34054 (local address 127.2.243.1:33659)
0807 06:34:14.106239 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:14.106241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:14.106250 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:34:14.106287 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:14.106287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:14.106308 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:14.106379 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:34:14.106595 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.107069 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.107885 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:14.108082 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:14.108142 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:14.108282 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:34:14.108377 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:34:14.108440 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:14.108471 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 06:34:14.108520 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:34:14.110494 20220 raft_consensus.cc:1261] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 1 FOLLOWER]: Refusing update from remote peer cc311562db914f44b5c26d48ab2eb63d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:34:14.110549 20438 raft_consensus.cc:1261] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 1 FOLLOWER]: Refusing update from remote peer cc311562db914f44b5c26d48ab2eb63d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:34:14.110957 20548 consensus_queue.cc:1035] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d3ef0ec7f948c0bbfa713f36d3a7d2" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:34:14.111119 20553 consensus_queue.cc:1035] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace195ad27424977b51cf90d312698bf" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:34:14.113505 20558 mvcc.cc:205] Tried to move back new op lower bound from 7315801924033912832 to 7315801923877560320. Current Snapshot: MvccSnapshot[applied={T|T < 7315801924033912832}]
I20260807 06:34:14.114055 20559 mvcc.cc:205] Tried to move back new op lower bound from 7315801924033912832 to 7315801923877560320. Current Snapshot: MvccSnapshot[applied={T|T < 7315801924033912832}]
I20260807 06:34:14.114179 20560 mvcc.cc:205] Tried to move back new op lower bound from 7315801924033912832 to 7315801923877560320. Current Snapshot: MvccSnapshot[applied={T|T < 7315801924033912832}]
I20260807 06:34:14.115723 19767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:34:14.115804 19767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:34:14.116636 19767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f7db2e2cf424388bc4663f0ce520f5f: Sending DeleteTablet for 3 replicas of tablet 6da5ccbcd94e453194e61d86baec7492
I20260807 06:34:14.117069 20396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6da5ccbcd94e453194e61d86baec7492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:34:14 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:44952
I20260807 06:34:14.117069 20188 tablet_service.cc:1514] Processing DeleteTablet for tablet 6da5ccbcd94e453194e61d86baec7492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:34:14 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:51688
I20260807 06:34:14.117120 19969 tablet_service.cc:1514] Processing DeleteTablet for tablet 6da5ccbcd94e453194e61d86baec7492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:34:14 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:34038
I20260807 06:34:14.117290 20575 tablet_replica.cc:330] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: stopping tablet replica
I20260807 06:34:14.117381 20575 raft_consensus.cc:2229] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:34:14.117378 20576 tablet_replica.cc:330] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d: stopping tablet replica
I20260807 06:34:14.117478 20575 raft_consensus.cc:2258] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:34:14.117491 20576 raft_consensus.cc:2229] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:34:14.117604 20576 raft_consensus.cc:2258] T 6da5ccbcd94e453194e61d86baec7492 P cc311562db914f44b5c26d48ab2eb63d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:34:14.117691 20574 tablet_replica.cc:330] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: stopping tablet replica
I20260807 06:34:14.117761 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 19856
I20260807 06:34:14.120397 20574 raft_consensus.cc:2229] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:34:14.120587 20574 raft_consensus.cc:2258] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:34:14.122511 20575 ts_tablet_manager.cc:1905] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:34:14.125062 20575 ts_tablet_manager.cc:1918] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:34:14.125133 20575 log.cc:1196] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/6da5ccbcd94e453194e61d86baec7492
I20260807 06:34:14.125379 20575 ts_tablet_manager.cc:1939] T 6da5ccbcd94e453194e61d86baec7492 P 81d3ef0ec7f948c0bbfa713f36d3a7d2: Deleting consensus metadata
I20260807 06:34:14.126914 19748 catalog_manager.cc:4841] TS 81d3ef0ec7f948c0bbfa713f36d3a7d2 (127.2.243.3:39225): tablet 6da5ccbcd94e453194e61d86baec7492 (table test-table [id=fc5a435bb34943d49687c2d391a1b71d]) successfully deleted
I20260807 06:34:14.127121 20574 ts_tablet_manager.cc:1905] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:34:14.128825 20574 ts_tablet_manager.cc:1918] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:34:14.128887 20574 log.cc:1196] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/6da5ccbcd94e453194e61d86baec7492
I20260807 06:34:14.129130 20574 ts_tablet_manager.cc:1939] T 6da5ccbcd94e453194e61d86baec7492 P ace195ad27424977b51cf90d312698bf: Deleting consensus metadata
I20260807 06:34:14.130174 19748 catalog_manager.cc:4841] TS ace195ad27424977b51cf90d312698bf (127.2.243.2:36539): tablet 6da5ccbcd94e453194e61d86baec7492 (table test-table [id=fc5a435bb34943d49687c2d391a1b71d]) successfully deleted
W20260807 06:34:14.142336 19746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:33659 (error 108)
I20260807 06:34:14.142624 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 20083
I20260807 06:34:14.144996 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:14.144839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:33659 (local address 127.0.0.1:34056)
0807 06:34:14.144897 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 06:34:14.144938 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:33659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:34:14.146358 19746 catalog_manager.cc:4584] TS cc311562db914f44b5c26d48ab2eb63d (127.2.243.1:33659): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6da5ccbcd94e453194e61d86baec7492: Network error: Client connection negotiation failed: client connection to 127.2.243.1:33659: connect: Connection refused (error 111)
I20260807 06:34:14.155203 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 20298
I20260807 06:34:14.163281 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 19717
2026-08-07T06:34:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2320 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:34:14 dist-test-slave-rwrg krb5kdc[20583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:34:14 dist-test-slave-rwrg krb5kdc[20583](info): set up 2 sockets
Aug 07 06:34:14 dist-test-slave-rwrg krb5kdc[20583](info): commencing operation
krb5kdc: starting...
W20260807 06:34:15.503685 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:34:15 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:34:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:34:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/rangeradmin_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/rangeradmin_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/rangerlookup_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/rangerlookup_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
I20260807 06:34:15.679643 3020 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTdeU2g/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/postgres -l logfile start
2026-08-07 06:34:18.263 UTC [20629] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 06:34:18.263 UTC [20629] LOG: listening on IPv4 address "127.2.243.20", port 32845
2026-08-07 06:34:18.267 UTC [20629] LOG: listening on Unix socket "/tmp/.s.PGSQL.32845"
2026-08-07 06:34:18.302 UTC [20633] LOG: database system was shut down at 2026-08-07 06:34:16 UTC
2026-08-07 06:34:18.306 UTC [20629] LOG: database system is ready to accept connections
I20260807 06:34:19.532539 3020 mini_postgres.cc:96] Postgres bound to 32845
2026-08-07 06:34:19.540 UTC [20649] FATAL: database "slave" does not exist
127.2.243.20:32845 - accepting connections
I20260807 06:34:19.541195 3020 mini_ranger.cc:161] Starting Ranger...
I20260807 06:34:19.557251 3020 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 06:34:20.037700 3020 mini_ranger.cc:87] Created ranger Postgres database
I20260807 06:34:20.037772 3020 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-admin
2026-08-07 06:34:20,364 [I] DB FLAVOR :POSTGRES
2026-08-07 06:34:20,364 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:34:20,364 [I] Checking connection..
2026-08-07 06:34:20,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 06:34:20,628 [I] Checking connection passed.
2026-08-07 06:34:20,628 [I] --------- Verifying version history table ---------
2026-08-07 06:34:20,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:34:20,906 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:34:20,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:34:21,180 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:34:21,180 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 06:34:21,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 06:34:21.442 UTC [20747] WARNING: there is no transaction in progress
2026-08-07 06:34:21,451 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 06:34:21,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:34:21,714 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 06:34:21,714 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:34:21,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 06:34:21,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 06:34:22,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rwrg.c.gcp-upstream.internal','N') ;"
2026-08-07 06:34:22,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:34:22,777 [I] Table x_portal_user does not exist in database ranger
2026-08-07 06:34:22,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:34:23,061 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 06:34:23,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:34:23,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:34:23,582 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 06:34:23,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 06:34:24.549 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.566 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.583 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.598 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.615 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.635 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.693 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.701 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.716 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.727 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.738 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.753 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.764 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.771 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.778 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:24.786 UTC [20921] WARNING: there is no transaction in progress
2026-08-07 06:34:25,445 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 06:34:25,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 06:34:25,693 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 06:34:25,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:34:25,945 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 06:34:25,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:34:26,186 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 06:34:26,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:34:26,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:34:26,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rwrg.c.gcp-upstream.internal';"
2026-08-07 06:34:26,935 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 06:34:26,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 06:34:27,198 [I] DB_PATCHES have already been applied
I20260807 06:34:27.203948 3020 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTdeU2g/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 06:34:27.204033 3020 mini_ranger.cc:193] Using host: 127.2.243.20
I20260807 06:34:27.207688 3020 mini_ranger.cc:227] Ranger admin URL: http://127.2.243.20:34421
Aug 07, 2026 6:34:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:34:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:34:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:34:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:34:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:34:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:34:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 06:34:28.593303 3020 mini_ranger.cc:160] Time spent starting Ranger: real 9.052s user 0.000s sys 0.011s
Aug 07, 2026 6:34:48 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:34:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:34:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:34:48 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:34:49 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 06:34:49.741211 3020 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/rangerkms_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/rangerkms_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.243.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.243.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 06:34:49 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084489, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 06:34:49.827011 3020 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 06:34:49.836833 3020 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 06:34:50.684103 3020 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 06:34:50.684192 3020 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms
I20260807 06:34:50.684217 3020 mini_ranger_kms.cc:227] Using postgres at 127.2.243.20:32845
2026-08-07 06:34:50,913 [I] DB FLAVOR :POSTGRES
2026-08-07 06:34:50,914 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:34:50,914 [I] Checking connection
2026-08-07 06:34:50,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 06:34:51,285 [I] connection success
2026-08-07 06:34:51,285 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 06:34:51,285 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 06:34:51,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 06:34:51,559 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 06:34:51,559 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:34:51,559 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 06:34:51,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:32845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 06:34:51,849 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 06:34:52.377048 3020 mini_ranger_kms.cc:310] Created kms service
I20260807 06:34:52.564760 3020 mini_ranger_kms.cc:326] Created kudu user
I20260807 06:34:52.610373 3020 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 06:34:52 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084489, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:34:53.009707 3020 mini_ranger_kms.cc:384] Added ranger policy
I20260807 06:34:53.009848 3020 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 06:34:53.009872 3020 mini_ranger_kms.cc:240] Using host: 127.2.243.20
I20260807 06:34:53.013800 3020 mini_ranger_kms.cc:276] Ranger KMS PID: 21203
I20260807 06:34:53.013933 3020 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.243.20:43187
Aug 07, 2026 6:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 06:34:55 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084495, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.243.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:34:55.866607 3020 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.040s user 0.002s sys 0.198s
I20260807 06:34:55.866729 3020 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 06:34:55.866814 3020 mini_ranger_kms.cc:401] 127.2.243.20:43187/kms/v1/keys
Aug 07 06:34:55 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084495, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.243.20@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
Aug 07 06:34:56 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084495, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.243.20@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
Aug 07 06:34:58 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:34:58.741314 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33907
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:33907
--ranger_config_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:43187/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:34:58.836186 21293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:58.836465 21293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:58.836531 21293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:58.837740 21293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:34:58.837779 21293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:58.837795 21293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:58.837807 21293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:34:58.837821 21293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:34:58.839372 21293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.243.20:43187/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:33907
--ranger_config_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:58.839560 21293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:58.839738 21293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:34:58.845209 21293 server_base.cc:1034] running on GCE node
W20260807 06:34:58.845358 21307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:58.845460 21301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:58.845504 21302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:58.845803 21293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:58.846153 21293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:58.847321 21293 hybrid_clock.cc:648] HybridClock initialized: now 1786084498847310 us; error 32 us; skew 500 ppm
Aug 07 06:34:58 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084498, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:34:58.853404 21293 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:34:58.853925 21293 webserver.cc:460] Webserver started at http://127.2.243.62:39965/ using document root <none> and password file <none>
I20260807 06:34:58.854166 21293 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:58.854223 21293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:58.854374 21293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:34:58 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084498, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:34:58.912644 21293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "f416f80aaf694f429a1792358fcee9c4"
format_stamp: "Formatted at 2026-08-07 06:34:58 on dist-test-slave-rwrg"
server_key: "6fbbffbe264d086c87f47679237b1eb3"
server_key_iv: "b203446b5ab1c4e49f1f56ae01526c17"
server_key_version: "kuduclusterkey@0"
I20260807 06:34:58.913266 21293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "f416f80aaf694f429a1792358fcee9c4"
format_stamp: "Formatted at 2026-08-07 06:34:58 on dist-test-slave-rwrg"
server_key: "6fbbffbe264d086c87f47679237b1eb3"
server_key_iv: "b203446b5ab1c4e49f1f56ae01526c17"
server_key_version: "kuduclusterkey@0"
I20260807 06:34:58.915452 21293 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 06:34:58.933302 21317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:58.933579 21293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:34:58.933713 21293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "f416f80aaf694f429a1792358fcee9c4"
format_stamp: "Formatted at 2026-08-07 06:34:58 on dist-test-slave-rwrg"
server_key: "6fbbffbe264d086c87f47679237b1eb3"
server_key_iv: "b203446b5ab1c4e49f1f56ae01526c17"
server_key_version: "kuduclusterkey@0"
I20260807 06:34:58.933836 21293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:34:58.949090 21293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:34:58.951193 21293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:34:58.951352 21293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:34:58.957633 21293 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:33907
I20260807 06:34:58.957692 21393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:33907 every 8 connection(s)
I20260807 06:34:58.958109 21293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:34:58.958298 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 21293
I20260807 06:34:58.958443 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:34:58.958740 3020 external_mini_cluster.cc:1428] Setting key 4591d5940c672246adde5c5309513499
I20260807 06:34:58.959594 21394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:34:58.963307 21394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4: Bootstrap starting.
I20260807 06:34:58.964001 21394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4: Neither blocks nor log segments found. Creating new log.
I20260807 06:34:58.964365 21394 log.cc:824] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:34:58.965404 21394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4: No bootstrap required, opened a new log
Aug 07 06:34:58 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:34:58.967744 21394 raft_consensus.cc:348] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f416f80aaf694f429a1792358fcee9c4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33907 } }
I20260807 06:34:58.967972 21394 raft_consensus.cc:374] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:34:58.968020 21394 raft_consensus.cc:729] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f416f80aaf694f429a1792358fcee9c4, State: Initialized, Role: FOLLOWER
I20260807 06:34:58.968171 21394 consensus_queue.cc:260] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f416f80aaf694f429a1792358fcee9c4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33907 } }
I20260807 06:34:58.968245 21394 raft_consensus.cc:388] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:34:58.968268 21394 raft_consensus.cc:482] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:34:58.968340 21394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:34:58.969074 21394 raft_consensus.cc:504] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f416f80aaf694f429a1792358fcee9c4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33907 } }
I20260807 06:34:58.969209 21394 leader_election.cc:302] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f416f80aaf694f429a1792358fcee9c4; no voters:
I20260807 06:34:58.969446 21394 leader_election.cc:288] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:34:58.969525 21405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:34:58.969882 21394 sys_catalog.cc:564] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:34:58.969851 21405 raft_consensus.cc:686] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [term 1 LEADER]: Becoming Leader. State: Replica: f416f80aaf694f429a1792358fcee9c4, State: Running, Role: LEADER
I20260807 06:34:58.970225 21405 consensus_queue.cc:237] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f416f80aaf694f429a1792358fcee9c4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33907 } }
I20260807 06:34:58.970659 21406 sys_catalog.cc:455] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f416f80aaf694f429a1792358fcee9c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f416f80aaf694f429a1792358fcee9c4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33907 } } }
I20260807 06:34:58.970777 21406 sys_catalog.cc:458] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:58.971226 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:34:58.959376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.2.243.62:33907)
0807 06:34:58.959606 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 06:34:58.959617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:34:58.959719 (+ 102us) server_negotiation.cc:407] Connection header received
0807 06:34:58.960377 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:34:58.960387 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:34:58.960461 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:34:58.960650 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:34:58.961420 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:58.962919 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:58.963813 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:34:58.964942 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:34:58.965069 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:34:58.967561 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:34:58.967565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:34:58.967565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:34:58.967576 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:34:58.969411 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:58.969841 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:58.969844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:58.969844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:58.969998 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:34:58.970241 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:34:58.970242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:34:58.970242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:34:58.970613 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:34:58.970783 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 06:34:58.970869 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 06:34:58.970989 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":125,"threads_started":1}
I20260807 06:34:58.971299 21407 sys_catalog.cc:455] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f416f80aaf694f429a1792358fcee9c4. Latest consensus state: current_term: 1 leader_uuid: "f416f80aaf694f429a1792358fcee9c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f416f80aaf694f429a1792358fcee9c4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33907 } } }
I20260807 06:34:58.971386 21407 sys_catalog.cc:458] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:34:58.971789 21415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:34:58.972352 21415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:34:58.972783 21394 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:34:58.978772 21415 catalog_manager.cc:1347] Generated new cluster ID: 555c4e7857fb47a1819b4013bf5cb85f
I20260807 06:34:58.978852 21415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:34:58.988349 21415 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:34:58.989251 21415 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:34:58.992772 21415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4: Generated new TSK 0
I20260807 06:34:58.992944 21415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:34:59.822733 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33907
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:34:59.923712 21463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:34:59.923940 21463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:34:59.923983 21463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:34:59.925323 21463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:34:59.925477 21463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:34:59.925560 21463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:34:59.927563 21463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.243.20:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:34:59.927851 21463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:34:59.928193 21463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:34:59.934917 21480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:59.934917 21474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:34:59.934918 21477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:34:59.935617 21463 server_base.cc:1034] running on GCE node
I20260807 06:34:59.935905 21463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:34:59.936340 21463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:34:59.937549 21463 hybrid_clock.cc:648] HybridClock initialized: now 1786084499937527 us; error 51 us; skew 500 ppm
Aug 07 06:34:59 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:34:59.940804 21463 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:34:59.941290 21463 webserver.cc:460] Webserver started at http://127.2.243.1:39107/ using document root <none> and password file <none>
I20260807 06:34:59.941514 21463 fs_manager.cc:359] Metadata directory not provided
I20260807 06:34:59.941584 21463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:34:59.941749 21463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:34:59 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:34:59.955312 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "3340273670ac4137abcdd72a5564d6cc"
format_stamp: "Formatted at 2026-08-07 06:34:59 on dist-test-slave-rwrg"
server_key: "34b91372b41cf13c244bef45f44a73a3"
server_key_iv: "440b55849c04b9ca7e59f99d66a4cad1"
server_key_version: "kuduclusterkey@0"
I20260807 06:34:59.955854 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "3340273670ac4137abcdd72a5564d6cc"
format_stamp: "Formatted at 2026-08-07 06:34:59 on dist-test-slave-rwrg"
server_key: "34b91372b41cf13c244bef45f44a73a3"
server_key_iv: "440b55849c04b9ca7e59f99d66a4cad1"
server_key_version: "kuduclusterkey@0"
I20260807 06:34:59.961006 21463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260807 06:34:59.978634 21492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:34:59.980283 21463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 06:34:59.980439 21463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "3340273670ac4137abcdd72a5564d6cc"
format_stamp: "Formatted at 2026-08-07 06:34:59 on dist-test-slave-rwrg"
server_key: "34b91372b41cf13c244bef45f44a73a3"
server_key_iv: "440b55849c04b9ca7e59f99d66a4cad1"
server_key_version: "kuduclusterkey@0"
I20260807 06:34:59.980605 21463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:00.005949 21463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:00.008703 21463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:00.008937 21463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:00.009290 21463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:00.009792 21463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:00.009893 21463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.010015 21463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:00.010106 21463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.017484 21463 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:36557
I20260807 06:35:00.018007 21463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:35:00.021631 21657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:36557 every 8 connection(s)
I20260807 06:35:00.026852 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 21463
I20260807 06:35:00.026997 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:35:00.027248 3020 external_mini_cluster.cc:1428] Setting key 1e9339589e36db160e61c56fde605989
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:00.033325 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.022189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33907 (local address 127.2.243.1:39299)
0807 06:35:00.022376 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.022422 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.022907 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.023222 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.023225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.023427 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:00.023640 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.023644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.024667 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.024668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.025588 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.025593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.026050 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.026052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.026185 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.027232 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:00.027243 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:00.029104 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:00.031313 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:00.031322 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:00.031325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:00.031672 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:00.031939 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:00.031940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:00.031941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:00.032105 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:00.032464 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:00.032472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:00.032711 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.032860 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.033059 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260807 06:35:00.033947 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.022246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:39299 (local address 127.2.243.62:33907)
0807 06:35:00.022373 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.022376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.022533 (+ 157us) server_negotiation.cc:407] Connection header received
0807 06:35:00.023048 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.023049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.023064 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.023166 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:00.023740 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.024575 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.025716 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.025947 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.026032 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.029212 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:00.029214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:00.029214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:00.029238 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:00.031203 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.031776 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.031778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.031778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.031855 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.032208 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.032209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.032209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.032363 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:00.032433 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.033730 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.033832 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260807 06:35:00.034793 21658 heartbeater.cc:347] Connected to a master server at 127.2.243.62:33907
I20260807 06:35:00.034914 21658 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:00.035130 21658 heartbeater.cc:511] Master 127.2.243.62:33907 requested a full tablet report, sending...
I20260807 06:35:00.035789 21339 ts_manager.cc:194] Registered new tserver with Master: 3340273670ac4137abcdd72a5564d6cc (127.2.243.1:36557)
I20260807 06:35:00.036834 21339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:39299
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:35:00.063390 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33907
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:00.181272 21664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:00.181499 21664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:00.181538 21664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:00.182744 21664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:00.182809 21664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:00.182854 21664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:35:00.184393 21664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.243.20:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:00.184674 21664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:00.184918 21664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:00.191452 21682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:00.191383 21673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:00.191754 21677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:00.192869 21664 server_base.cc:1034] running on GCE node
I20260807 06:35:00.193123 21664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:00.193372 21664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:00.200767 21664 hybrid_clock.cc:648] HybridClock initialized: now 1786084500200742 us; error 37 us; skew 500 ppm
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:00.203775 21664 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:35:00.204339 21664 webserver.cc:460] Webserver started at http://127.2.243.2:45981/ using document root <none> and password file <none>
I20260807 06:35:00.204640 21664 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:00.204713 21664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:00.204860 21664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:00.230347 21664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "3c688d37ed5b41ec8877db868023fd5c"
format_stamp: "Formatted at 2026-08-07 06:35:00 on dist-test-slave-rwrg"
server_key: "2f365e2dcb02a581b20b2d3b81f04660"
server_key_iv: "235734ccb06b8555e52f07dbf2bee17e"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:00.231040 21664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "3c688d37ed5b41ec8877db868023fd5c"
format_stamp: "Formatted at 2026-08-07 06:35:00 on dist-test-slave-rwrg"
server_key: "2f365e2dcb02a581b20b2d3b81f04660"
server_key_iv: "235734ccb06b8555e52f07dbf2bee17e"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:00.233439 21664 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 06:35:00.251570 21696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.251816 21664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 06:35:00.251904 21664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "3c688d37ed5b41ec8877db868023fd5c"
format_stamp: "Formatted at 2026-08-07 06:35:00 on dist-test-slave-rwrg"
server_key: "2f365e2dcb02a581b20b2d3b81f04660"
server_key_iv: "235734ccb06b8555e52f07dbf2bee17e"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:00.252002 21664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:00.282685 21664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:00.285483 21664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:00.285694 21664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:00.285986 21664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:00.286432 21664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:00.286520 21664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.286563 21664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:00.286644 21664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.299904 21664 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:38373
I20260807 06:35:00.300033 21865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:38373 every 8 connection(s)
I20260807 06:35:00.300439 21664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:35:00.307029 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 21664
I20260807 06:35:00.307155 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:35:00.307350 3020 external_mini_cluster.cc:1428] Setting key 051c7407e1288fab98210711abda6c4a
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:00.313239 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.301823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:58181 (local address 127.2.243.62:33907)
0807 06:35:00.301884 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.301886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.301999 (+ 113us) server_negotiation.cc:407] Connection header received
0807 06:35:00.303007 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.303008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.303029 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.303103 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:00.304076 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.304889 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.306486 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.306815 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.306886 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.309777 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:00.309778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:00.309779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:00.309786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:00.311491 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.311951 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.311952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.311952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.312010 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.312299 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.312300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.312301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.312431 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:00.312472 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.313047 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.313152 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 06:35:00.313881 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.301442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33907 (local address 127.2.243.2:58181)
0807 06:35:00.301800 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.301851 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.302876 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.303174 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.303179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.303562 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:00.303968 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.303975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.304975 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.304977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.306387 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.306390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.306955 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.306956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.307113 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.307796 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:00.307806 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:00.309664 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:00.311585 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:00.311592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:00.311594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:00.311863 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:00.312095 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:00.312095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:00.312096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:00.312223 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:00.312564 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:00.312570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:00.312834 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.313393 (+ 559us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.313592 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":168,"threads_started":1}
I20260807 06:35:00.314435 21866 heartbeater.cc:347] Connected to a master server at 127.2.243.62:33907
I20260807 06:35:00.314534 21866 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:00.314713 21866 heartbeater.cc:511] Master 127.2.243.62:33907 requested a full tablet report, sending...
I20260807 06:35:00.315092 21339 ts_manager.cc:194] Registered new tserver with Master: 3c688d37ed5b41ec8877db868023fd5c (127.2.243.2:38373)
I20260807 06:35:00.315599 21339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:58181
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:35:00.349257 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33907
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:00.493784 21874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:00.494055 21874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:00.494088 21874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:00.495731 21874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:00.495785 21874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:00.495822 21874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:35:00.498024 21874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.243.20:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:33907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:00.498250 21874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:00.498494 21874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:00.506430 21881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:00.506649 21882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:00.506752 21884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:00.508005 21874 server_base.cc:1034] running on GCE node
I20260807 06:35:00.508229 21874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:00.508468 21874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:00.509652 21874 hybrid_clock.cc:648] HybridClock initialized: now 1786084500509630 us; error 33 us; skew 500 ppm
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:00.512780 21874 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:35:00.513239 21874 webserver.cc:460] Webserver started at http://127.2.243.3:35261/ using document root <none> and password file <none>
I20260807 06:35:00.513475 21874 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:00.513530 21874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:00.513650 21874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:00.527580 21874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "9830a1a622a34d7288aefbdb97661262"
format_stamp: "Formatted at 2026-08-07 06:35:00 on dist-test-slave-rwrg"
server_key: "18d7727393e8a8a076f3de65263ba760"
server_key_iv: "e0a9e1d20804bee295ddd6e2e0f54db1"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:00.528031 21874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "9830a1a622a34d7288aefbdb97661262"
format_stamp: "Formatted at 2026-08-07 06:35:00 on dist-test-slave-rwrg"
server_key: "18d7727393e8a8a076f3de65263ba760"
server_key_iv: "e0a9e1d20804bee295ddd6e2e0f54db1"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:00.535444 21874 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.002s
I20260807 06:35:00.558868 21894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.559098 21874 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.001s
I20260807 06:35:00.559197 21874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "9830a1a622a34d7288aefbdb97661262"
format_stamp: "Formatted at 2026-08-07 06:35:00 on dist-test-slave-rwrg"
server_key: "18d7727393e8a8a076f3de65263ba760"
server_key_iv: "e0a9e1d20804bee295ddd6e2e0f54db1"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:00.559305 21874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:00.581629 21874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:00.584213 21874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:00.584355 21874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:00.584595 21874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:00.585017 21874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:00.585062 21874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.585101 21874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:00.585139 21874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:00.598860 21874 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:33089
I20260807 06:35:00.599066 22061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:33089 every 8 connection(s)
I20260807 06:35:00.599539 21874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:35:00.607409 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 21874
I20260807 06:35:00.607513 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:35:00.607694 3020 external_mini_cluster.cc:1428] Setting key 32fd5859b9c2828a5cd9f44f0c118d4a
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:00.612071 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.601074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:57163 (local address 127.2.243.62:33907)
0807 06:35:00.601142 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.601144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.601165 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:35:00.601849 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.601851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.601863 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.601933 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:00.602772 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.603615 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.605121 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.605439 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.605538 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.608689 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:00.608690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:00.608690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:00.608698 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:00.610512 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.611052 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.611053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.611054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.611111 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.611363 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.611364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.611364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.611498 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:00.611537 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.611903 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.611982 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:35:00.612583 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.600699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33907 (local address 127.2.243.3:57163)
0807 06:35:00.600900 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.600978 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.601724 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.602003 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.602007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.602297 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:00.602655 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.602662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.603734 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.603735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.605029 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.605032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.605538 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.605539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.605719 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.606460 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:00.606470 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:00.608588 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:00.610680 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:00.610687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:00.610689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:00.610971 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:00.611179 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:00.611180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:00.611181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:00.611297 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:00.611591 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:00.611596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:00.611762 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.612191 (+ 429us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.612338 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260807 06:35:00.612980 22062 heartbeater.cc:347] Connected to a master server at 127.2.243.62:33907
I20260807 06:35:00.613061 22062 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:00.613320 22062 heartbeater.cc:511] Master 127.2.243.62:33907 requested a full tablet report, sending...
I20260807 06:35:00.613686 21339 ts_manager.cc:194] Registered new tserver with Master: 9830a1a622a34d7288aefbdb97661262 (127.2.243.3:33089)
I20260807 06:35:00.614229 21339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:57163
I20260807 06:35:00.620949 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:35:00.633284 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.623198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41404 (local address 127.2.243.62:33907)
0807 06:35:00.623282 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.623284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.623433 (+ 149us) server_negotiation.cc:407] Connection header received
0807 06:35:00.623539 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.623540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.623549 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.623605 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:00.624959 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.625626 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.627332 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.627649 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.627720 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.629891 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:00.629892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:00.629893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:00.629901 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:00.631704 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.632249 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.632250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.632251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.632313 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:00.632638 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:00.632639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:00.632639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:00.632769 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:00.632870 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.633122 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.633198 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:35:00.636001 21339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:35:00.638363 21339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:35:00.649346 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.641439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.2.243.1:36557)
0807 06:35:00.641610 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.641616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.643044 (+ 1428us) server_negotiation.cc:407] Connection header received
0807 06:35:00.643087 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.643090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.643108 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.643664 (+ 556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:00.644242 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.645810 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.647393 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.648435 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.648836 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.648914 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.649200 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.649264 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260807 06:35:00.649636 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.641149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:36557 (local address 127.0.0.1:44426)
0807 06:35:00.641679 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.642916 (+ 1237us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.642981 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.643860 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.643864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.643867 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:00.643964 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.643966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.645934 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.645935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.647286 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.647290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.648980 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.648981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.649066 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.649083 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.649466 (+ 383us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.649550 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":98,"threads_started":1}
I20260807 06:35:00.654491 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.641155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:33089 (local address 127.0.0.1:39748)
0807 06:35:00.649707 (+ 8552us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.649719 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.649769 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.650169 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.650170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.650172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:00.650203 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.650204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.651991 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.651992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.653144 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.653147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.654270 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.654271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.654327 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.654332 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.654385 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.654421 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8542,"thread_start_us":156,"threads_started":1}
I20260807 06:35:00.655251 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.641553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.2.243.3:33089)
0807 06:35:00.646088 (+ 4535us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.646093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.649837 (+ 3744us) server_negotiation.cc:407] Connection header received
0807 06:35:00.649930 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.649933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.649952 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.650033 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:00.650284 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.651897 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.653267 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.654036 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.654109 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.654233 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.654715 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.654755 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4474,"thread_start_us":140,"threads_started":1}
I20260807 06:35:00.656045 21562 tablet_service.cc:1467] Processing CreateTablet for tablet 0e7a16f82ada4df69cd2179cc8d38e10 (DEFAULT_TABLE table=test-table [id=580352ed96a0473ea644b348e600802a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:00.656102 21942 tablet_service.cc:1467] Processing CreateTablet for tablet 0e7a16f82ada4df69cd2179cc8d38e10 (DEFAULT_TABLE table=test-table [id=580352ed96a0473ea644b348e600802a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:00.656450 21942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e7a16f82ada4df69cd2179cc8d38e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:00.656437 21562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e7a16f82ada4df69cd2179cc8d38e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:00.662990 22095 tablet_bootstrap.cc:492] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc: Bootstrap starting.
I20260807 06:35:00.664067 22095 tablet_bootstrap.cc:654] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:00.664861 22095 log.cc:824] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:00.665984 22095 tablet_bootstrap.cc:492] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc: No bootstrap required, opened a new log
I20260807 06:35:00.666142 22095 ts_tablet_manager.cc:1397] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:35:00.666595 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.641149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:38373 (local address 127.0.0.1:51182)
0807 06:35:00.642671 (+ 1522us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.642717 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.642790 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.655096 (+ 12306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.655097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.655100 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:00.655141 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.655143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.662885 (+ 7742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.662886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.664101 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.664105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.666353 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.666354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.666423 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.666429 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.666459 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.666516 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1456,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260807 06:35:00.667155 22091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.641446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.2.243.2:38373)
0807 06:35:00.654896 (+ 13450us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.654901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.654915 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:35:00.654954 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.654957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.654973 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.655098 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:00.661629 (+ 6531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.662760 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.665283 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.666230 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.666778 (+ 548us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.666949 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.666978 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.667020 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13386,"thread_start_us":75,"threads_started":1}
I20260807 06:35:00.667788 21738 tablet_service.cc:1467] Processing CreateTablet for tablet 0e7a16f82ada4df69cd2179cc8d38e10 (DEFAULT_TABLE table=test-table [id=580352ed96a0473ea644b348e600802a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:00.668141 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e7a16f82ada4df69cd2179cc8d38e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:00.668746 22097 tablet_bootstrap.cc:492] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Bootstrap starting.
I20260807 06:35:00.670372 22097 tablet_bootstrap.cc:654] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:00.670384 22095 raft_consensus.cc:348] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.670601 22095 raft_consensus.cc:374] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:00.671108 22095 raft_consensus.cc:729] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3340273670ac4137abcdd72a5564d6cc, State: Initialized, Role: FOLLOWER
I20260807 06:35:00.671330 22095 consensus_queue.cc:260] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.671715 22095 ts_tablet_manager.cc:1428] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260807 06:35:00.671854 21658 heartbeater.cc:503] Master 127.2.243.62:33907 was elected leader, sending a full tablet report...
I20260807 06:35:00.673148 22097 log.cc:824] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:00.681036 22097 tablet_bootstrap.cc:492] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: No bootstrap required, opened a new log
I20260807 06:35:00.681136 22097 ts_tablet_manager.cc:1397] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260807 06:35:00.683981 22097 raft_consensus.cc:348] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.684188 22097 raft_consensus.cc:374] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:00.684234 22097 raft_consensus.cc:729] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9830a1a622a34d7288aefbdb97661262, State: Initialized, Role: FOLLOWER
I20260807 06:35:00.684435 22097 consensus_queue.cc:260] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.684792 22097 ts_tablet_manager.cc:1428] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 06:35:00.684918 22062 heartbeater.cc:503] Master 127.2.243.62:33907 was elected leader, sending a full tablet report...
I20260807 06:35:00.685547 22103 tablet_bootstrap.cc:492] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Bootstrap starting.
I20260807 06:35:00.687780 22103 tablet_bootstrap.cc:654] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:00.688313 22103 log.cc:824] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:00.689422 22103 tablet_bootstrap.cc:492] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: No bootstrap required, opened a new log
I20260807 06:35:00.689497 22103 ts_tablet_manager.cc:1397] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:35:00.691870 22103 raft_consensus.cc:348] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.692113 22103 raft_consensus.cc:374] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:00.692152 22103 raft_consensus.cc:729] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c688d37ed5b41ec8877db868023fd5c, State: Initialized, Role: FOLLOWER
I20260807 06:35:00.692343 22103 consensus_queue.cc:260] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.692673 22103 ts_tablet_manager.cc:1428] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:35:00.692792 21866 heartbeater.cc:503] Master 127.2.243.62:33907 was elected leader, sending a full tablet report...
I20260807 06:35:00.760766 22111 raft_consensus.cc:482] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:35:00.760910 22111 raft_consensus.cc:504] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.761299 22111 leader_election.cc:288] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9830a1a622a34d7288aefbdb97661262 (127.2.243.3:33089), 3340273670ac4137abcdd72a5564d6cc (127.2.243.1:36557)
I20260807 06:35:00.765506 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.761523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:36557 (local address 127.2.243.2:51567)
0807 06:35:00.761635 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.761645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.761684 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.762316 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.762317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.762319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:00.762384 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.762386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.763262 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.763264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.764469 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.764472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.765293 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.765294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.765362 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.765365 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.765405 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.765436 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 06:35:00.766005 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.761776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:51567 (local address 127.2.243.1:36557)
0807 06:35:00.761826 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.761828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.761835 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:35:00.761861 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.761861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.761872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.761915 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:00.762464 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.763158 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.764625 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.765190 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.765252 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.765262 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.765899 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.765940 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:35:00.766248 21592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a16f82ada4df69cd2179cc8d38e10" candidate_uuid: "3c688d37ed5b41ec8877db868023fd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3340273670ac4137abcdd72a5564d6cc" is_pre_election: true
I20260807 06:35:00.766396 21592 raft_consensus.cc:2454] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c688d37ed5b41ec8877db868023fd5c in term 0.
I20260807 06:35:00.766644 21701 leader_election.cc:302] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3340273670ac4137abcdd72a5564d6cc, 3c688d37ed5b41ec8877db868023fd5c; no voters:
I20260807 06:35:00.766772 22111 raft_consensus.cc:2781] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:35:00.766813 22111 raft_consensus.cc:482] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:35:00.766839 22111 raft_consensus.cc:3037] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:00.767722 22111 raft_consensus.cc:504] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.767867 22111 leader_election.cc:288] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [CANDIDATE]: Term 1 election: Requested vote from peers 9830a1a622a34d7288aefbdb97661262 (127.2.243.3:33089), 3340273670ac4137abcdd72a5564d6cc (127.2.243.1:36557)
I20260807 06:35:00.768052 21592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a16f82ada4df69cd2179cc8d38e10" candidate_uuid: "3c688d37ed5b41ec8877db868023fd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3340273670ac4137abcdd72a5564d6cc"
I20260807 06:35:00.768127 21592 raft_consensus.cc:3037] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:00.769001 21592 raft_consensus.cc:2454] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c688d37ed5b41ec8877db868023fd5c in term 1.
I20260807 06:35:00.769189 21701 leader_election.cc:302] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3340273670ac4137abcdd72a5564d6cc, 3c688d37ed5b41ec8877db868023fd5c; no voters:
I20260807 06:35:00.769287 22111 raft_consensus.cc:2781] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:00.769433 22111 raft_consensus.cc:686] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 1 LEADER]: Becoming Leader. State: Replica: 3c688d37ed5b41ec8877db868023fd5c, State: Running, Role: LEADER
I20260807 06:35:00.769615 22111 consensus_queue.cc:237] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } }
I20260807 06:35:00.770431 21339 catalog_manager.cc:5495] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c688d37ed5b41ec8877db868023fd5c (127.2.243.2). New cstate: current_term: 1 leader_uuid: "3c688d37ed5b41ec8877db868023fd5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c688d37ed5b41ec8877db868023fd5c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38373 } health_report { overall_health: HEALTHY } } }
W20260807 06:35:00.772423 21659 tablet.cc:2369] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:35:00.773829 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.762041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:33089 (local address 127.2.243.2:42741)
0807 06:35:00.765594 (+ 3553us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.765604 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:00.765639 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:00.770924 (+ 5285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:00.770925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:00.770927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:00.770963 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.770965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.771879 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.771880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.772873 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:00.772876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.773641 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.773642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:00.773694 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.773698 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:35:00.773732 (+ 34us) client_negotiation.cc:240] Negotiation successful
0807 06:35:00.773765 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3543,"thread_start_us":98,"threads_started":1}
I20260807 06:35:00.774148 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.762250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:42741 (local address 127.2.243.3:33089)
0807 06:35:00.762287 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.762289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.765686 (+ 3397us) server_negotiation.cc:407] Connection header received
0807 06:35:00.765732 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.765733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.765744 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.765786 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:00.771064 (+ 5278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.771763 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.773013 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.773524 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.773599 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.773610 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.774052 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.774090 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:35:00.774681 21993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a16f82ada4df69cd2179cc8d38e10" candidate_uuid: "3c688d37ed5b41ec8877db868023fd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9830a1a622a34d7288aefbdb97661262" is_pre_election: true
I20260807 06:35:00.774816 21993 raft_consensus.cc:2454] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c688d37ed5b41ec8877db868023fd5c in term 0.
I20260807 06:35:00.774911 21994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e7a16f82ada4df69cd2179cc8d38e10" candidate_uuid: "3c688d37ed5b41ec8877db868023fd5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9830a1a622a34d7288aefbdb97661262"
I20260807 06:35:00.774955 21994 raft_consensus.cc:3037] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:00.775811 21994 raft_consensus.cc:2454] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c688d37ed5b41ec8877db868023fd5c in term 1.
I20260807 06:35:00.792855 22091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.789416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.2.243.2:38373)
0807 06:35:00.789466 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:00.789468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:00.789476 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:35:00.789525 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:00.789526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:00.789547 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:00.789630 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:35:00.789858 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.790601 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.791707 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:00.791981 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:00.792046 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:00.792252 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:35:00.792380 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:35:00.792454 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:00.792723 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 06:35:00.792754 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:35:00.795204 21592 raft_consensus.cc:1261] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3340273670ac4137abcdd72a5564d6cc [term 1 FOLLOWER]: Refusing update from remote peer 3c688d37ed5b41ec8877db868023fd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:00.795641 22114 consensus_queue.cc:1035] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3340273670ac4137abcdd72a5564d6cc" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:00.798460 21994 raft_consensus.cc:1261] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 1 FOLLOWER]: Refusing update from remote peer 3c688d37ed5b41ec8877db868023fd5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:00.798832 22114 consensus_queue.cc:1035] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9830a1a622a34d7288aefbdb97661262" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 33089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:00.805825 21339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:00.805924 21339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:00.806963 21339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f416f80aaf694f429a1792358fcee9c4: Sending DeleteTablet for 3 replicas of tablet 0e7a16f82ada4df69cd2179cc8d38e10
I20260807 06:35:00.807505 21562 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e7a16f82ada4df69cd2179cc8d38e10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:00 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:44426
I20260807 06:35:00.807991 21738 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e7a16f82ada4df69cd2179cc8d38e10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:00 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:51182
I20260807 06:35:00.808081 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 21463
I20260807 06:35:00.814419 21942 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e7a16f82ada4df69cd2179cc8d38e10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:00 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:39748
I20260807 06:35:00.818348 22117 mvcc.cc:205] Tried to move back new op lower bound from 7315802115253739520 to 7315802115153063936. Current Snapshot: MvccSnapshot[applied={T|T < 7315802115253739520 or (T in {7315802115253739520})}]
I20260807 06:35:00.820709 22136 tablet_replica.cc:330] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: stopping tablet replica
I20260807 06:35:00.820860 22136 raft_consensus.cc:2229] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:00.820948 22136 pending_rounds.cc:62] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Trying to abort 2 pending ops.
I20260807 06:35:00.820982 22136 pending_rounds.cc:68] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315802115253739520 op_type: WRITE_OP write_request { tablet_id: "0e7a16f82ada4df69cd2179cc8d38e10" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\300\373\325\323\006\"8\n\ntest-admin\022*\n 580352ed96a0473ea644b348e600802a\020\001\030\001 \001(\001" signature: "<redacted>""c sM%\322o{\005?\244\212\333\216\210\256\354\007\335r\372f\365\022\201\024Gi\261\264P\312\322:\202\304t\340V\211T\031%\200\216N\265\271s\3571\\\024!J\342I8<\232\021WP\027" signing_key_seq_num: 0 } } request_id { client_id: "8476d6d4ceca4f9fb590f1d682186a4c" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260807 06:35:00.821164 22136 pending_rounds.cc:68] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315802115153063936 op_type: NO_OP noop_request { }
I20260807 06:35:00.821213 22136 raft_consensus.cc:2866] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 06:35:00.821240 22136 raft_consensus.cc:2258] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:00.832710 22128 tablet_replica.cc:330] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: stopping tablet replica
I20260807 06:35:00.832871 22128 raft_consensus.cc:2229] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:35:00.832996 22128 raft_consensus.cc:2258] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:00.833796 22128 ts_tablet_manager.cc:1905] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:35:00.835752 21321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:36557 (error 108)
I20260807 06:35:00.835963 22128 ts_tablet_manager.cc:1918] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:00.836035 22128 log.cc:1196] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/0e7a16f82ada4df69cd2179cc8d38e10
I20260807 06:35:00.836313 22128 ts_tablet_manager.cc:1939] T 0e7a16f82ada4df69cd2179cc8d38e10 P 3c688d37ed5b41ec8877db868023fd5c: Deleting consensus metadata
I20260807 06:35:00.836802 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 21664
I20260807 06:35:00.836792 21318 catalog_manager.cc:4841] TS 3c688d37ed5b41ec8877db868023fd5c (127.2.243.2:38373): tablet 0e7a16f82ada4df69cd2179cc8d38e10 (table test-table [id=580352ed96a0473ea644b348e600802a]) successfully deleted
I20260807 06:35:00.843843 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:00.843682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:36557 (local address 127.0.0.1:44442)
0807 06:35:00.843748 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:00.843804 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:36557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 06:35:00.844123 21321 catalog_manager.cc:4584] TS 3340273670ac4137abcdd72a5564d6cc (127.2.243.1:36557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e7a16f82ada4df69cd2179cc8d38e10: Network error: Client connection negotiation failed: client connection to 127.2.243.1:36557: connect: Connection refused (error 111)
W20260807 06:35:00.847936 22124 consensus_queue.cc:1159] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9830a1a622a34d7288aefbdb97661262" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260807 06:35:00.850832 22136 ts_tablet_manager.cc:1905] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:35:00.852963 22136 ts_tablet_manager.cc:1918] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:00.853060 22136 log.cc:1196] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/0e7a16f82ada4df69cd2179cc8d38e10
I20260807 06:35:00.853719 22136 ts_tablet_manager.cc:1939] T 0e7a16f82ada4df69cd2179cc8d38e10 P 9830a1a622a34d7288aefbdb97661262: Deleting consensus metadata
I20260807 06:35:00.854302 21322 catalog_manager.cc:4841] TS 9830a1a622a34d7288aefbdb97661262 (127.2.243.3:33089): tablet 0e7a16f82ada4df69cd2179cc8d38e10 (table test-table [id=580352ed96a0473ea644b348e600802a]) successfully deleted
W20260807 06:35:00.854527 21898 connection.cc:535] server connection from 127.2.243.2:42741 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:35:00.855224 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 21874
I20260807 06:35:00.870445 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 21293
Aug 07 06:35:00 dist-test-slave-rwrg krb5kdc[20583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084500, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:00.883154 3020 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 06:35:01.227747 3020 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 06:35:01.227852 3020 mini_ranger.cc:66] Stopping Ranger...
I20260807 06:35:01.572196 3020 mini_ranger.cc:68] Stopped Ranger
2026-08-07 06:35:01.572 UTC [20629] LOG: received smart shutdown request
2026-08-07 06:35:01.576 UTC [20629] LOG: background worker "logical replication launcher" (PID 20646) exited with exit code 1
2026-08-07 06:35:01.576 UTC [20634] LOG: shutting down
2026-08-07 06:35:01.745 UTC [20629] LOG: database system is shut down
2026-08-07T06:35:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47688 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:35:01 dist-test-slave-rwrg krb5kdc[22154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:35:01 dist-test-slave-rwrg krb5kdc[22154](info): set up 2 sockets
Aug 07 06:35:01 dist-test-slave-rwrg krb5kdc[22154](info): commencing operation
krb5kdc: starting...
W20260807 06:35:03.104992 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:35:03 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:35:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:35:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/rangeradmin_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/rangeradmin_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/rangerlookup_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/rangerlookup_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
I20260807 06:35:03.274740 3020 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTdeU2g/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/postgres -l logfile start
2026-08-07 06:35:05.745 UTC [22211] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 06:35:05.746 UTC [22211] LOG: listening on IPv4 address "127.2.243.20", port 39431
2026-08-07 06:35:05.749 UTC [22211] LOG: listening on Unix socket "/tmp/.s.PGSQL.39431"
2026-08-07 06:35:05.769 UTC [22216] LOG: database system was shut down at 2026-08-07 06:35:04 UTC
2026-08-07 06:35:05.773 UTC [22211] LOG: database system is ready to accept connections
I20260807 06:35:06.936950 3020 mini_postgres.cc:96] Postgres bound to 39431
2026-08-07 06:35:06.944 UTC [22229] FATAL: database "slave" does not exist
127.2.243.20:39431 - accepting connections
I20260807 06:35:06.945222 3020 mini_ranger.cc:161] Starting Ranger...
I20260807 06:35:06.960397 3020 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 06:35:07.424966 3020 mini_ranger.cc:87] Created ranger Postgres database
I20260807 06:35:07.425055 3020 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-admin
2026-08-07 06:35:07,704 [I] DB FLAVOR :POSTGRES
2026-08-07 06:35:07,704 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:35:07,704 [I] Checking connection..
2026-08-07 06:35:07,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 06:35:07,976 [I] Checking connection passed.
2026-08-07 06:35:07,976 [I] --------- Verifying version history table ---------
2026-08-07 06:35:07,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:35:08,212 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:35:08,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:35:08,478 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:35:08,478 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 06:35:08,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 06:35:08.737 UTC [22323] WARNING: there is no transaction in progress
2026-08-07 06:35:08,746 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 06:35:08,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:35:09,012 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 06:35:09,012 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:35:09,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 06:35:09,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 06:35:09,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rwrg.c.gcp-upstream.internal','N') ;"
2026-08-07 06:35:09,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:35:10,034 [I] Table x_portal_user does not exist in database ranger
2026-08-07 06:35:10,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:35:10,272 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 06:35:10,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:35:10,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:35:10,787 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 06:35:10,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 06:35:11.785 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.801 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.817 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.832 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.847 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.864 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.917 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.924 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.938 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.949 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.959 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.973 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.982 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.988 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:11.995 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:12.001 UTC [22508] WARNING: there is no transaction in progress
2026-08-07 06:35:12,644 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 06:35:12,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 06:35:12,901 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 06:35:12,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:35:13,139 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 06:35:13,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:35:13,404 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 06:35:13,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:35:13,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:35:13,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rwrg.c.gcp-upstream.internal';"
2026-08-07 06:35:14,184 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 06:35:14,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 06:35:14,449 [I] DB_PATCHES have already been applied
I20260807 06:35:14.455511 3020 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTdeU2g/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 06:35:14.455579 3020 mini_ranger.cc:193] Using host: 127.2.243.20
I20260807 06:35:14.459427 3020 mini_ranger.cc:227] Ranger admin URL: http://127.2.243.20:56017
Aug 07, 2026 6:35:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:35:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:35:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:35:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:35:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:35:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:35:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 06:35:15.822580 3020 mini_ranger.cc:160] Time spent starting Ranger: real 8.877s user 0.006s sys 0.004s
Aug 07, 2026 6:35:35 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:35:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:35:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:35:35 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:35:36 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 06:35:36.626385 3020 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.243.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.243.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/rangerkms_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.243.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/rangerkms_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.243.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.243.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.243.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/HTTP_127.2.243.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 06:35:36 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084536, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 06:35:36.717133 3020 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 06:35:36.727263 3020 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 06:35:37.589949 3020 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 06:35:37.590073 3020 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms
I20260807 06:35:37.590097 3020 mini_ranger_kms.cc:227] Using postgres at 127.2.243.20:39431
2026-08-07 06:35:37,773 [I] DB FLAVOR :POSTGRES
2026-08-07 06:35:37,773 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:35:37,773 [I] Checking connection
2026-08-07 06:35:37,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 06:35:38,036 [I] connection success
2026-08-07 06:35:38,036 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 06:35:38,036 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 06:35:38,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 06:35:38,290 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 06:35:38,291 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:35:38,291 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 06:35:38,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.243.20:39431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 06:35:38,545 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 06:35:39.045027 3020 mini_ranger_kms.cc:310] Created kms service
I20260807 06:35:39.219640 3020 mini_ranger_kms.cc:326] Created kudu user
I20260807 06:35:39.266322 3020 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 06:35:39 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084536, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:39.658708 3020 mini_ranger_kms.cc:384] Added ranger policy
I20260807 06:35:39.658839 3020 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTdeU2g/build/release/bin/postgresql.jar:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTdeU2g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 06:35:39.658859 3020 mini_ranger_kms.cc:240] Using host: 127.2.243.20
I20260807 06:35:39.663094 3020 mini_ranger_kms.cc:276] Ranger KMS PID: 22796
I20260807 06:35:39.663185 3020 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.243.20:48361
Aug 07, 2026 6:35:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:35:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:35:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:35:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:35:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:35:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 06:35:42 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.243.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 06:35:42 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.243.20@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:42.486157 3020 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.769s user 0.005s sys 0.162s
I20260807 06:35:42.486264 3020 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 06:35:42.486306 3020 mini_ranger_kms.cc:401] 127.2.243.20:48361/kms/v1/keys
Aug 07 06:35:42 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.243.20@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
Aug 07 06:35:45 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:35:45.231719 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:32811
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:32811
--ranger_config_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:45.314134 22867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:45.314368 22867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:45.314407 22867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:45.315567 22867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:45.315631 22867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:45.315653 22867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:45.315675 22867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:45.315706 22867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:45.317294 22867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.243.20:48361/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:32811
--ranger_config_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:32811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:45.317564 22867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:45.317781 22867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:35:45.323206 22867 server_base.cc:1034] running on GCE node
W20260807 06:35:45.323275 22872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:45.323257 22873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:45.323282 22876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:45.323658 22867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:45.323933 22867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:45.325089 22867 hybrid_clock.cc:648] HybridClock initialized: now 1786084545325077 us; error 28 us; skew 500 ppm
Aug 07 06:35:45 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084545, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:45.327726 22867 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:35:45.328233 22867 webserver.cc:460] Webserver started at http://127.2.243.62:41929/ using document root <none> and password file <none>
I20260807 06:35:45.328467 22867 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:45.328558 22867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:45.328720 22867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:35:45 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084545, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:45.375061 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "2064d76940bf4986b910a8d76897da4b"
format_stamp: "Formatted at 2026-08-07 06:35:45 on dist-test-slave-rwrg"
server_key: "0b6e5ce7265ef658f4af7797d7520ca1"
server_key_iv: "a0f42e006c5517aecb01b54274ab2d8f"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:45.375478 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "2064d76940bf4986b910a8d76897da4b"
format_stamp: "Formatted at 2026-08-07 06:35:45 on dist-test-slave-rwrg"
server_key: "0b6e5ce7265ef658f4af7797d7520ca1"
server_key_iv: "a0f42e006c5517aecb01b54274ab2d8f"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:45.377422 22867 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 06:35:45.391757 22887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:45.392046 22867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:35:45.392167 22867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "2064d76940bf4986b910a8d76897da4b"
format_stamp: "Formatted at 2026-08-07 06:35:45 on dist-test-slave-rwrg"
server_key: "0b6e5ce7265ef658f4af7797d7520ca1"
server_key_iv: "a0f42e006c5517aecb01b54274ab2d8f"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:45.392287 22867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:45.415033 22867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:45.417179 22867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:45.417300 22867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:45.421661 22867 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:32811
I20260807 06:35:45.421747 22968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:32811 every 8 connection(s)
I20260807 06:35:45.422113 22867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:35:45.422854 22971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:45.425323 22971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Bootstrap starting.
I20260807 06:35:45.425940 22971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:45.426234 22971 log.cc:824] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:45.426930 22971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: No bootstrap required, opened a new log
I20260807 06:35:45.428515 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 22867
I20260807 06:35:45.428632 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:35:45.428853 3020 external_mini_cluster.cc:1428] Setting key 214476cd0c74dc72de855dbdfd78268b
I20260807 06:35:45.429311 22971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:45.429536 22971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:45.429625 22971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2064d76940bf4986b910a8d76897da4b, State: Initialized, Role: FOLLOWER
I20260807 06:35:45.429770 22971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:45.429850 22971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:35:45.429881 22971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:35:45.429926 22971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:45.430514 22971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:45.430601 22971 leader_election.cc:302] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2064d76940bf4986b910a8d76897da4b; no voters:
I20260807 06:35:45.430807 22971 leader_election.cc:288] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:35:45.430837 22978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:45.430959 22978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 LEADER]: Becoming Leader. State: Replica: 2064d76940bf4986b910a8d76897da4b, State: Running, Role: LEADER
I20260807 06:35:45.431088 22971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:45.431103 22978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:45.431443 22979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2064d76940bf4986b910a8d76897da4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } } }
I20260807 06:35:45.431542 22979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:45.431433 22978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2064d76940bf4986b910a8d76897da4b. Latest consensus state: current_term: 1 leader_uuid: "2064d76940bf4986b910a8d76897da4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } } }
I20260807 06:35:45.431807 22978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:45.432166 22987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:35:45.432845 22987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:35:45.433207 22971 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 07 06:35:45 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:45.443794 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:45.429432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.2.243.62:32811)
0807 06:35:45.429652 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:45.429662 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:45.429696 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:35:45.429809 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:45.429816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:45.429872 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:45.430034 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:45.430729 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:45.432167 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:45.433400 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:45.435648 (+ 2248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:45.436009 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:45.439015 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:45.439020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:45.439020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:45.439032 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:45.441886 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:45.442362 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:45.442366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:45.442366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:45.442495 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:45.442734 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:45.442735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:45.442736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:45.443077 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:45.443242 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:45.443382 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 06:35:45.443553 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":99,"threads_started":1}
I20260807 06:35:45.445217 22987 catalog_manager.cc:1347] Generated new cluster ID: 9cd9b6fc89d142b6b182c1867fb18c91
I20260807 06:35:45.445272 22987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:35:45.458475 22987 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:35:45.459308 22987 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:35:45.466436 22987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Generated new TSK 0
I20260807 06:35:45.466646 22987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:35:46.143808 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:32811
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:46.235297 23033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:46.235546 23033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:46.235585 23033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:46.236814 23033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:46.236872 23033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:46.236915 23033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:35:46.238368 23033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.243.20:48361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:32811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:46.238619 23033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:46.238842 23033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:46.244117 23042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:46.244287 23043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:46.244375 23033 server_base.cc:1034] running on GCE node
W20260807 06:35:46.244110 23045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:46.244688 23033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:46.244959 23033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:46.246120 23033 hybrid_clock.cc:648] HybridClock initialized: now 1786084546246098 us; error 39 us; skew 500 ppm
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:46.248823 23033 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:35:46.249323 23033 webserver.cc:460] Webserver started at http://127.2.243.1:33401/ using document root <none> and password file <none>
I20260807 06:35:46.249619 23033 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:46.249701 23033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:46.249861 23033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:46.263437 23033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "428e31e9107f40ffb6bde5f500e7b933"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "565300cc6178563981f6816075125152"
server_key_iv: "1a2402981301e102cf4d99debc5f3ef9"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.263906 23033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "428e31e9107f40ffb6bde5f500e7b933"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "565300cc6178563981f6816075125152"
server_key_iv: "1a2402981301e102cf4d99debc5f3ef9"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.268889 23033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 06:35:46.284775 23055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.284950 23033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:46.285034 23033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "428e31e9107f40ffb6bde5f500e7b933"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "565300cc6178563981f6816075125152"
server_key_iv: "1a2402981301e102cf4d99debc5f3ef9"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.285121 23033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:46.298036 23033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:46.300202 23033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:46.300350 23033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:46.300648 23033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:46.301024 23033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:46.301079 23033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.301116 23033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:46.301143 23033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.309152 23033 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:39345
I20260807 06:35:46.309197 23238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:39345 every 8 connection(s)
I20260807 06:35:46.309556 23033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:35:46.310506 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 23033
I20260807 06:35:46.310629 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:35:46.310868 3020 external_mini_cluster.cc:1428] Setting key 7c792ae64b527c13abdcab4a5f387b78
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:46.321300 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:46.310348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:32811 (local address 127.2.243.1:56917)
0807 06:35:46.310620 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:46.310661 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:46.311272 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:46.311630 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:46.311635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:46.311867 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:46.312070 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:46.312074 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.312894 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.312895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:46.314035 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:46.314038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.314603 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.314604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:46.314786 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:46.315403 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:46.315413 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:46.317254 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:46.319429 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:46.319437 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:46.319440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:46.319797 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:46.320035 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:46.320036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:46.320036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:46.320177 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:46.320519 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:46.320527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:46.320746 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 06:35:46.320871 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 06:35:46.321053 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
I20260807 06:35:46.322275 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:46.310363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:56917 (local address 127.2.243.62:32811)
0807 06:35:46.310518 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:46.310521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:46.310746 (+ 225us) server_negotiation.cc:407] Connection header received
0807 06:35:46.311424 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:46.311426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:46.311438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:46.311502 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:46.312187 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.312800 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.314155 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.314458 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.314523 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:46.317374 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:46.317377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:46.317377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:46.317385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:46.319328 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:46.319889 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:46.319890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:46.319891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:46.319961 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:46.320258 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:46.320259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:46.320260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:46.320405 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:46.320464 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:46.321863 (+ 1399us) server_negotiation.cc:299] Negotiation successful
0807 06:35:46.321954 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260807 06:35:46.322994 23241 heartbeater.cc:347] Connected to a master server at 127.2.243.62:32811
I20260807 06:35:46.323112 23241 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:46.323307 23241 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:46.323977 22911 ts_manager.cc:194] Registered new tserver with Master: 428e31e9107f40ffb6bde5f500e7b933 (127.2.243.1:39345)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
I20260807 06:35:46.325070 22911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:56917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:35:46.341799 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:32811
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:46.459056 23249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:46.459342 23249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:46.459378 23249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:46.461282 23249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:46.461345 23249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:46.461388 23249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:35:46.463814 23249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.243.20:48361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:32811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:46.464073 23249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:46.464346 23249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:46.471063 23262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:46.471722 23249 server_base.cc:1034] running on GCE node
W20260807 06:35:46.471951 23259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:46.472131 23258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:46.472362 23249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:46.472678 23249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:46.473858 23249 hybrid_clock.cc:648] HybridClock initialized: now 1786084546473828 us; error 46 us; skew 500 ppm
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:46.477226 23249 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:35:46.477793 23249 webserver.cc:460] Webserver started at http://127.2.243.2:44637/ using document root <none> and password file <none>
I20260807 06:35:46.478070 23249 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:46.478142 23249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:46.478288 23249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:46.494436 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "758eead109c04bc5b2ad63c5e5f51681"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "5ae7a85fa6faa894119bf040f6b3c5b7"
server_key_iv: "efeea985e4abaf1f37317f49be060c48"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.494984 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "758eead109c04bc5b2ad63c5e5f51681"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "5ae7a85fa6faa894119bf040f6b3c5b7"
server_key_iv: "efeea985e4abaf1f37317f49be060c48"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.499701 23249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 06:35:46.525939 23271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.526141 23249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:35:46.526237 23249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "758eead109c04bc5b2ad63c5e5f51681"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "5ae7a85fa6faa894119bf040f6b3c5b7"
server_key_iv: "efeea985e4abaf1f37317f49be060c48"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.526338 23249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:46.545727 23249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:46.547864 23249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:46.548030 23249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:46.548342 23249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:46.548805 23249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:46.548893 23249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.548934 23249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:46.548965 23249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.556227 23249 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:41929
I20260807 06:35:46.556694 23249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:35:46.559180 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 23249
I20260807 06:35:46.559319 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:35:46.559537 3020 external_mini_cluster.cc:1428] Setting key 70cd82758cd082be3bb1da6adc99ef9d
I20260807 06:35:46.561156 23455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:41929 every 8 connection(s)
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
I20260807 06:35:46.573383 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:46.562183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:43941 (local address 127.2.243.62:32811)
0807 06:35:46.562734 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:46.562736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:46.564881 (+ 2145us) server_negotiation.cc:407] Connection header received
0807 06:35:46.565418 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:46.565419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:46.565432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:46.565476 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:46.566083 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.566651 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.567668 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.567891 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.567981 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:46.570363 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:46.570370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:46.570370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:46.570377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:46.571809 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:46.572252 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:46.572253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:46.572254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:46.572312 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:46.572636 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:46.572638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:46.572638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:46.572786 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:46.572836 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:46.573200 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 06:35:46.573292 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516}
I20260807 06:35:46.573534 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:46.561832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:32811 (local address 127.2.243.2:43941)
0807 06:35:46.564754 (+ 2922us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:46.564791 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:46.565305 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:46.565577 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:46.565580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:46.565783 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:46.565969 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:46.565973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.566747 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.566749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:46.567574 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:46.567576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.568006 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.568007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:46.568142 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:46.568952 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:46.568961 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:46.570255 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:46.571914 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:46.571922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:46.571924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:46.572165 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:46.572383 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:46.572384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:46.572384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:46.572545 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:46.572887 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:46.572894 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:46.573058 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 06:35:46.573163 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:35:46.573310 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2885,"thread_start_us":121,"threads_started":1}
I20260807 06:35:46.574060 23457 heartbeater.cc:347] Connected to a master server at 127.2.243.62:32811
I20260807 06:35:46.574182 23457 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:46.574365 23457 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:46.574802 22911 ts_manager.cc:194] Registered new tserver with Master: 758eead109c04bc5b2ad63c5e5f51681 (127.2.243.2:41929)
I20260807 06:35:46.575274 22911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:43941
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:35:46.597679 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:32811
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:46.724810 23464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:46.725189 23464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:46.725282 23464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:46.727074 23464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:46.727210 23464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:46.727300 23464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:35:46.729740 23464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.243.20:48361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:32811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:46.730217 23464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:46.730546 23464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:46.737215 23474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:46.737587 23477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:46.737877 23479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:46.740073 23464 server_base.cc:1034] running on GCE node
I20260807 06:35:46.740329 23464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:46.740691 23464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:46.741890 23464 hybrid_clock.cc:648] HybridClock initialized: now 1786084546741857 us; error 49 us; skew 500 ppm
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:46.745721 23464 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:35:46.746351 23464 webserver.cc:460] Webserver started at http://127.2.243.3:38255/ using document root <none> and password file <none>
I20260807 06:35:46.746671 23464 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:46.746757 23464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:46.746919 23464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:46.763155 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "2aa1f72e96064030a86840b7689da44d"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "1b09c453a5dfa5534f0ed566edb34908"
server_key_iv: "72c9429e7b8dd5e52a31e841e7c8ce3e"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.763701 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "2aa1f72e96064030a86840b7689da44d"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "1b09c453a5dfa5534f0ed566edb34908"
server_key_iv: "72c9429e7b8dd5e52a31e841e7c8ce3e"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.769058 23464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 06:35:46.787106 23493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.787395 23464 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260807 06:35:46.787501 23464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "2aa1f72e96064030a86840b7689da44d"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "1b09c453a5dfa5534f0ed566edb34908"
server_key_iv: "72c9429e7b8dd5e52a31e841e7c8ce3e"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:46.787602 23464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:46.819375 23464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:46.827023 23464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:46.827219 23464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:46.827517 23464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:46.828042 23464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:46.828096 23464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.828131 23464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:46.828161 23464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:46.835664 23464 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:39115
I20260807 06:35:46.836189 23464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:35:46.837812 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 23464
I20260807 06:35:46.837917 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:35:46.838128 3020 external_mini_cluster.cc:1428] Setting key 3123ee798ff58f796524ff4cc7996322
I20260807 06:35:46.841643 23679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:39115 every 8 connection(s)
Aug 07 06:35:46 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:46.855089 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:46.842688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:50529 (local address 127.2.243.62:32811)
0807 06:35:46.842735 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:46.842737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:46.842941 (+ 204us) server_negotiation.cc:407] Connection header received
0807 06:35:46.843685 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:46.843687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:46.843699 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:46.843732 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:46.844662 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.845492 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.847215 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.847554 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.847606 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:46.851273 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:46.851275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:46.851276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:46.851284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:46.853279 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:46.853824 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:46.853826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:46.853826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:46.853892 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:46.854215 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:46.854216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:46.854216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:46.854392 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:46.854438 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:46.854884 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 06:35:46.854984 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:35:46.855540 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:46.842375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:32811 (local address 127.2.243.3:50529)
0807 06:35:46.842811 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:46.842858 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:46.843556 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:46.843917 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:46.843921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:46.844195 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:46.844504 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:46.844545 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.845629 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.845630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:46.847096 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:46.847099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:46.847723 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:46.847724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:46.847879 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:46.848605 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:46.848616 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:46.851151 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:46.853387 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:46.853395 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:46.853398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:46.853721 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:46.853972 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:46.853973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:46.853973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:46.854128 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:46.854491 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:46.854498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:46.854689 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 06:35:46.854818 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 06:35:46.855005 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":110,"threads_started":1}
I20260807 06:35:46.856261 23680 heartbeater.cc:347] Connected to a master server at 127.2.243.62:32811
I20260807 06:35:46.856374 23680 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:46.856630 23680 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:46.857097 22911 ts_manager.cc:194] Registered new tserver with Master: 2aa1f72e96064030a86840b7689da44d (127.2.243.3:39115)
I20260807 06:35:46.857656 22911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:50529
I20260807 06:35:46.860580 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:35:46.861403 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 23033
I20260807 06:35:46.878144 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 23249
I20260807 06:35:46.894701 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 23464
I20260807 06:35:46.913321 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 22867
I20260807 06:35:46.925736 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:32811
--webserver_interface=127.2.243.62
--webserver_port=41929
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:32811
--ranger_config_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:47.007948 23684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:47.008153 23684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:47.008175 23684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:47.009351 23684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:47.009397 23684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:47.009411 23684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:47.009424 23684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:47.009436 23684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:47.011104 23684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:32811
--ranger_config_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:32811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:47.011340 23684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:47.011577 23684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:47.016528 23695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.016623 23691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.016543 23693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:47.016758 23684 server_base.cc:1034] running on GCE node
I20260807 06:35:47.016983 23684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:47.017222 23684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:47.018396 23684 hybrid_clock.cc:648] HybridClock initialized: now 1786084547018381 us; error 43 us; skew 500 ppm
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:47.020817 23684 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:35:47.021271 23684 webserver.cc:460] Webserver started at http://127.2.243.62:41929/ using document root <none> and password file <none>
I20260807 06:35:47.021498 23684 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:47.021552 23684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:47.059854 23684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:47.061189 23707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.061427 23684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:47.061524 23684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "2064d76940bf4986b910a8d76897da4b"
format_stamp: "Formatted at 2026-08-07 06:35:45 on dist-test-slave-rwrg"
server_key: "0b6e5ce7265ef658f4af7797d7520ca1"
server_key_iv: "a0f42e006c5517aecb01b54274ab2d8f"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:47.061913 23684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:47.085851 23684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:47.088009 23684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:47.088178 23684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:47.092942 23684 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:32811
I20260807 06:35:47.093026 23798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:32811 every 8 connection(s)
I20260807 06:35:47.093421 23684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:35:47.094727 23799 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:35:47.095418 23799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Bootstrap starting.
I20260807 06:35:47.101792 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 23684
I20260807 06:35:47.102092 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:39345
--local_ip_for_outbound_sockets=127.2.243.1
--tserver_master_addrs=127.2.243.62:32811
--webserver_port=33401
--webserver_interface=127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:35:47.103694 23799 log.cc:824] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:47.105476 23799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:35:47.105772 23799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Bootstrap complete.
I20260807 06:35:47.108300 23799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:47.108585 23799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2064d76940bf4986b910a8d76897da4b, State: Initialized, Role: FOLLOWER
I20260807 06:35:47.108806 23799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:47.108868 23799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:35:47.108916 23799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:35:47.108963 23799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:35:47.109982 23799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:47.110078 23799 leader_election.cc:302] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2064d76940bf4986b910a8d76897da4b; no voters:
I20260807 06:35:47.110301 23799 leader_election.cc:288] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:35:47.110337 23805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:35:47.110447 23805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [term 2 LEADER]: Becoming Leader. State: Replica: 2064d76940bf4986b910a8d76897da4b, State: Running, Role: LEADER
I20260807 06:35:47.110570 23805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } }
I20260807 06:35:47.110579 23799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:47.110829 23807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2064d76940bf4986b910a8d76897da4b. Latest consensus state: current_term: 2 leader_uuid: "2064d76940bf4986b910a8d76897da4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } } }
I20260807 06:35:47.110900 23807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:47.111171 23806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2064d76940bf4986b910a8d76897da4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2064d76940bf4986b910a8d76897da4b" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32811 } } }
I20260807 06:35:47.111241 23806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:47.111310 23809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:35:47.111843 23809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:35:47.111982 23809 catalog_manager.cc:1259] Loaded cluster ID: 9cd9b6fc89d142b6b182c1867fb18c91
I20260807 06:35:47.112023 23809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:35:47.112375 23799 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:35:47.113195 23809 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:35:47.113371 23809 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Loaded TSK: 0
I20260807 06:35:47.113597 23809 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:35:47.193635 23803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:47.193997 23803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:47.194090 23803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:47.195688 23803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:47.195760 23803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:47.195804 23803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:35:47.197516 23803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:39345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=33401
--tserver_master_addrs=127.2.243.62:32811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:47.197760 23803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:47.198052 23803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:47.203181 23858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.203181 23863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.203505 23860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:47.204568 23803 server_base.cc:1034] running on GCE node
I20260807 06:35:47.204869 23803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:47.205148 23803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:47.206295 23803 hybrid_clock.cc:648] HybridClock initialized: now 1786084547206263 us; error 44 us; skew 500 ppm
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:47.209033 23803 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:35:47.209564 23803 webserver.cc:460] Webserver started at http://127.2.243.1:33401/ using document root <none> and password file <none>
I20260807 06:35:47.209826 23803 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:47.209892 23803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:47.228909 23803 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:47.229632 23879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.229826 23803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:47.229921 23803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "428e31e9107f40ffb6bde5f500e7b933"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "565300cc6178563981f6816075125152"
server_key_iv: "1a2402981301e102cf4d99debc5f3ef9"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:47.230297 23803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:47.238978 23803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:47.241218 23803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:47.241411 23803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:47.241782 23803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:47.242316 23803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:47.242414 23803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.242549 23803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:47.242626 23803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.250447 23803 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:39345
I20260807 06:35:47.250960 23803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:35:47.252965 24047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:39345 every 8 connection(s)
I20260807 06:35:47.258458 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 23803
I20260807 06:35:47.258750 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:41929
--local_ip_for_outbound_sockets=127.2.243.2
--tserver_master_addrs=127.2.243.62:32811
--webserver_port=44637
--webserver_interface=127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:47.269767 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:47.252702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:55137 (local address 127.2.243.62:32811)
0807 06:35:47.252925 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:47.252935 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:47.257528 (+ 4593us) server_negotiation.cc:407] Connection header received
0807 06:35:47.257682 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:47.257688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:47.257744 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:47.257880 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:47.258757 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.260184 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.261774 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.262401 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.262517 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:47.265536 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:47.265540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:47.265541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:47.265552 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:47.267306 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.267745 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.267749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.267749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.267838 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.268113 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.268114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.268114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.268347 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:47.268437 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:47.269364 (+ 927us) server_negotiation.cc:299] Negotiation successful
0807 06:35:47.269491 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":116,"threads_started":1}
I20260807 06:35:47.269170 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:47.252241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:32811 (local address 127.2.243.1:55137)
0807 06:35:47.256668 (+ 4427us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:47.256716 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:47.257345 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:47.258056 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:47.258060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:47.258313 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:47.258627 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:47.258633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.260325 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.260326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:47.261674 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:47.261677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.262635 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.262636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:47.262763 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:47.263355 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:47.263365 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:47.265406 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:47.267411 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:47.267419 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:47.267421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:47.267653 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:47.267916 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:47.267917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:47.267917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:47.268037 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:47.268431 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:47.268437 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:47.268627 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 06:35:47.268738 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 06:35:47.268902 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4381,"thread_start_us":94,"threads_started":1}
I20260807 06:35:47.277308 24048 heartbeater.cc:347] Connected to a master server at 127.2.243.62:32811
I20260807 06:35:47.277415 24048 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.277624 24048 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
W20260807 06:35:47.278726 24048 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:32811 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:35:47.278808 24048 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.278885 24048 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:47.279213 24048 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.279299 24048 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
W20260807 06:35:47.279637 24048 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:35:47.381315 24052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:47.381690 24052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:47.381786 24052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:47.383563 24052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:47.383642 24052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:47.383697 24052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:35:47.386091 24052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:41929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=44637
--tserver_master_addrs=127.2.243.62:32811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:47.386369 24052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:47.386708 24052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:47.393219 24063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.393817 24062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.393224 24065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:47.395116 24052 server_base.cc:1034] running on GCE node
I20260807 06:35:47.395359 24052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:47.395664 24052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:47.396831 24052 hybrid_clock.cc:648] HybridClock initialized: now 1786084547396809 us; error 37 us; skew 500 ppm
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:47.399973 24052 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:35:47.400496 24052 webserver.cc:460] Webserver started at http://127.2.243.2:44637/ using document root <none> and password file <none>
I20260807 06:35:47.400811 24052 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:47.400903 24052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:47.422824 24052 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.000s
I20260807 06:35:47.423693 24077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.423974 24052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:47.424080 24052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "758eead109c04bc5b2ad63c5e5f51681"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "5ae7a85fa6faa894119bf040f6b3c5b7"
server_key_iv: "efeea985e4abaf1f37317f49be060c48"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:47.424553 24052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:47.439532 24052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:47.441910 24052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:47.442091 24052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:47.442351 24052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:47.442772 24052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:47.442826 24052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.442858 24052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:47.442925 24052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.450697 24052 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:41929
I20260807 06:35:47.450778 24263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:41929 every 8 connection(s)
I20260807 06:35:47.451169 24052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:35:47.456959 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 24052
I20260807 06:35:47.457279 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.243.20:48361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:39115
--local_ip_for_outbound_sockets=127.2.243.3
--tserver_master_addrs=127.2.243.62:32811
--webserver_port=38255
--webserver_interface=127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:47.465031 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:47.451962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:55489 (local address 127.2.243.62:32811)
0807 06:35:47.452014 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:47.452016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:47.452264 (+ 248us) server_negotiation.cc:407] Connection header received
0807 06:35:47.452854 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:47.452855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:47.452873 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:47.452977 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:47.453574 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.454318 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.455764 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.456073 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.456165 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:47.461283 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:47.461285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:47.461286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:47.461296 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:47.463220 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.463803 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.463804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.463805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.463885 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.464250 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.464251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.464252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.464385 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:47.464442 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:47.464845 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 06:35:47.464959 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:35:47.465289 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:47.451968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:32811 (local address 127.2.243.2:55489)
0807 06:35:47.452134 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:47.452176 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:47.452742 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:47.453027 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:47.453031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:47.453252 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:47.453479 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:47.453483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.454424 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.454424 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:47.455681 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:47.455686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.456204 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.456206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:47.456353 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:47.459315 (+ 2962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:47.459326 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:47.461160 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:47.463333 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:47.463341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:47.463344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:47.463675 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:47.463970 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:47.463971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:47.463971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:47.464139 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:47.464464 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:47.464471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:47.464697 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 06:35:47.464819 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 06:35:47.464994 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260807 06:35:47.465847 24264 heartbeater.cc:347] Connected to a master server at 127.2.243.62:32811
I20260807 06:35:47.465977 24264 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.466184 24264 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
W20260807 06:35:47.466902 24264 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:32811 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:35:47.466988 24264 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.467067 24264 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:47.467499 24264 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.467615 24264 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
W20260807 06:35:47.467975 24264 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:35:47.572381 24269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:47.572777 24269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:47.572873 24269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:47.574645 24269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:47.574750 24269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:47.574826 24269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:35:47.577113 24269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.243.20:48361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:39115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=38255
--tserver_master_addrs=127.2.243.62:32811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:47.577520 24269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:47.577809 24269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:47.584298 24283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.584291 24277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:47.584290 24280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:47.585099 24269 server_base.cc:1034] running on GCE node
I20260807 06:35:47.585350 24269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:47.585632 24269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:47.586787 24269 hybrid_clock.cc:648] HybridClock initialized: now 1786084547586766 us; error 40 us; skew 500 ppm
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:47.591419 24269 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:35:47.591969 24269 webserver.cc:460] Webserver started at http://127.2.243.3:38255/ using document root <none> and password file <none>
I20260807 06:35:47.592337 24269 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:47.592482 24269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:47.612952 24269 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 06:35:47.614029 24301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.614326 24269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:47.614456 24269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "2aa1f72e96064030a86840b7689da44d"
format_stamp: "Formatted at 2026-08-07 06:35:46 on dist-test-slave-rwrg"
server_key: "1b09c453a5dfa5534f0ed566edb34908"
server_key_iv: "72c9429e7b8dd5e52a31e841e7c8ce3e"
server_key_version: "kuduclusterkey@0"
I20260807 06:35:47.614974 24269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:47.628731 24269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:47.632022 24269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:47.632210 24269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:47.632542 24269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:47.632968 24269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:47.633025 24269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.633098 24269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:47.633127 24269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:47.640959 24269 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:39115
I20260807 06:35:47.641376 24269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:35:47.642047 24484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:39115 every 8 connection(s)
I20260807 06:35:47.643831 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 24269
Aug 07 06:35:47 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:47.654862 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:47.642504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:46275 (local address 127.2.243.62:32811)
0807 06:35:47.642550 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:47.642552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:47.642564 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:35:47.643081 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:47.643083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:47.643097 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:47.643156 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:47.643784 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.644379 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.646119 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.646389 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.646428 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:47.649480 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:47.649481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:47.649482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:47.649490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:47.652565 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.653091 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.653092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.653092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.653168 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.653472 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.653473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.653473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.653606 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:47.653649 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:47.654683 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0807 06:35:47.654762 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:35:47.655682 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:47.642220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:32811 (local address 127.2.243.3:46275)
0807 06:35:47.642384 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:47.642427 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:47.642950 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:47.643228 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:47.643232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:47.643424 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:47.643653 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:47.643660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.644628 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.644629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:47.646015 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:47.646018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.646533 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.646534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:47.646683 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:47.647354 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:47.647364 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:47.649067 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:47.652665 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:47.652674 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:47.652677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:47.652994 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:47.653249 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:47.653249 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:47.653250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:47.653395 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:47.653699 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:47.653706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:47.653880 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 06:35:47.655259 (+ 1379us) client_negotiation.cc:240] Negotiation successful
0807 06:35:47.655439 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260807 06:35:47.656368 24485 heartbeater.cc:347] Connected to a master server at 127.2.243.62:32811
I20260807 06:35:47.656534 24485 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.656796 24485 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
W20260807 06:35:47.657477 24485 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:32811 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:35:47.657591 24485 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.657687 24485 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:47.657994 24485 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:47.658102 24485 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
W20260807 06:35:47.658397 24485 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:35:47.660180 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:47.644654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.2.243.62:32811)
0807 06:35:47.644981 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:47.644983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:47.645162 (+ 179us) server_negotiation.cc:407] Connection header received
0807 06:35:47.645184 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:47.645184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:47.645195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:47.645251 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:47.646039 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.646781 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.648012 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:47.648325 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:47.648424 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:47.650354 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:47.650355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:47.650356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:47.650363 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:47.654551 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.658956 (+ 4405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.658957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.658958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.659020 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:47.659255 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:47.659256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:47.659256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:47.659379 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:47.659939 (+ 560us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:47.659981 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 06:35:47.660068 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":76,"threads_started":1}
Aug 07 06:35:48 dist-test-slave-rwrg krb5kdc[22154](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786084548, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:48.279820 24048 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:48.280009 24048 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:48.280480 23735 ts_manager.cc:194] Registered new tserver with Master: 428e31e9107f40ffb6bde5f500e7b933 (127.2.243.1:39345)
I20260807 06:35:48.281561 23735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:55137
W20260807 06:35:48.451967 24258 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.2.243.2:41929
I20260807 06:35:48.468189 24264 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:48.468379 24264 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:48.468755 23735 ts_manager.cc:194] Registered new tserver with Master: 758eead109c04bc5b2ad63c5e5f51681 (127.2.243.2:41929)
I20260807 06:35:48.469208 23735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:55489
I20260807 06:35:48.658607 24485 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:48.658768 24485 heartbeater.cc:511] Master 127.2.243.62:32811 requested a full tablet report, sending...
I20260807 06:35:48.659104 23735 ts_manager.cc:194] Registered new tserver with Master: 2aa1f72e96064030a86840b7689da44d (127.2.243.3:39115)
I20260807 06:35:48.659632 23735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:46275
I20260807 06:35:48.669687 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:35:48 dist-test-slave-rwrg krb5kdc[22154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084548, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for HTTP/127.2.243.20@KRBTEST.COM
I20260807 06:35:48.684327 24518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.671035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.2.243.62:32811)
0807 06:35:48.676460 (+ 5425us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:48.676462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:48.676473 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:35:48.676529 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:48.676529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:48.676542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:48.676614 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:48.677295 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.678058 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.679057 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.679349 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.679405 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.680977 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:48.680979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:48.680979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:48.680987 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:48.682825 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:48.683216 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:48.683217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:48.683217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:48.683273 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:48.683446 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:48.683447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:48.683447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:48.683579 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:48.683667 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:48.684146 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 06:35:48.684225 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5399,"thread_start_us":76,"threads_started":1}
I20260807 06:35:48.686714 23735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:35:48.688889 23735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:35:48.705408 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.699847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:41929 (local address 127.0.0.1:46240)
0807 06:35:48.700666 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:48.700708 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:48.700783 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:48.701282 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:48.701285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:48.701288 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:48.701387 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.701390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.702635 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.702636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.703999 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.704002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.705103 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.705104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.705193 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.705208 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:35:48.705279 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:35:48.705328 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":88,"threads_started":1}
I20260807 06:35:48.706022 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.700726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46240 (local address 127.2.243.2:41929)
0807 06:35:48.700859 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:48.700864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:48.700873 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:35:48.700907 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:48.700915 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:48.700936 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:48.701472 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:48.701560 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.702555 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.704095 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.704924 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.705003 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.705065 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:48.705897 (+ 832us) server_negotiation.cc:299] Negotiation successful
0807 06:35:48.705943 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260807 06:35:48.706032 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.700087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39345 (local address 127.0.0.1:38156)
0807 06:35:48.701077 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:48.701088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:48.701141 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:48.701525 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:48.701526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:48.701528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:48.701573 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.701574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.703190 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.703191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.704354 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.704356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.705814 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.705815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.705881 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.705886 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:35:48.705915 (+ 29us) client_negotiation.cc:240] Negotiation successful
0807 06:35:48.705955 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"thread_start_us":58,"threads_started":1}
I20260807 06:35:48.706367 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.700476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.2.243.1:39345)
0807 06:35:48.701353 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:48.701358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:48.701367 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:35:48.701403 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:48.701405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:48.701419 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:48.701644 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:48.701716 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.703063 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.704972 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.705711 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.706140 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.706221 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:48.706248 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 06:35:48.706288 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":96,"threads_started":1}
I20260807 06:35:48.707036 24165 tablet_service.cc:1467] Processing CreateTablet for tablet 92ed4c19107042d787194849495f4266 (DEFAULT_TABLE table=test-table [id=c5441c23fae449eeab79fc36fd0d9733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:48.707121 23932 tablet_service.cc:1467] Processing CreateTablet for tablet 92ed4c19107042d787194849495f4266 (DEFAULT_TABLE table=test-table [id=c5441c23fae449eeab79fc36fd0d9733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:48.707469 24165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ed4c19107042d787194849495f4266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:48.707556 23932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ed4c19107042d787194849495f4266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:48.712680 24535 tablet_bootstrap.cc:492] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933: Bootstrap starting.
I20260807 06:35:48.713342 24535 tablet_bootstrap.cc:654] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:48.713627 24535 log.cc:824] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:48.714509 24535 tablet_bootstrap.cc:492] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933: No bootstrap required, opened a new log
I20260807 06:35:48.714566 24535 ts_tablet_manager.cc:1397] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:35:48.717118 24535 raft_consensus.cc:348] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.717413 24535 raft_consensus.cc:374] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:48.717495 24535 raft_consensus.cc:729] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e31e9107f40ffb6bde5f500e7b933, State: Initialized, Role: FOLLOWER
I20260807 06:35:48.718070 24535 consensus_queue.cc:260] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.721810 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.700231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:39115 (local address 127.0.0.1:60422)
0807 06:35:48.701214 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:48.701221 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:48.701261 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:48.702892 (+ 1631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:48.702893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:48.702896 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:48.702937 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.702938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.719397 (+ 16459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.719399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.720517 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.720520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.721575 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.721577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.721642 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.721648 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:35:48.721676 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 06:35:48.721722 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":93,"threads_started":1}
I20260807 06:35:48.722213 24535 ts_tablet_manager.cc:1428] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260807 06:35:48.722390 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.700888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.2.243.3:39115)
0807 06:35:48.701052 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:48.701056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:48.702389 (+ 1333us) server_negotiation.cc:407] Connection header received
0807 06:35:48.702430 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:48.702432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:48.702447 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:48.703022 (+ 575us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:48.703126 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.704150 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.720617 (+ 16467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.721470 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.722165 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.722240 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:48.722270 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 06:35:48.722316 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260807 06:35:48.722608 24048 heartbeater.cc:503] Master 127.2.243.62:32811 was elected leader, sending a full tablet report...
I20260807 06:35:48.723320 24374 tablet_service.cc:1467] Processing CreateTablet for tablet 92ed4c19107042d787194849495f4266 (DEFAULT_TABLE table=test-table [id=c5441c23fae449eeab79fc36fd0d9733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:48.723721 24374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ed4c19107042d787194849495f4266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:48.727329 24540 tablet_bootstrap.cc:492] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Bootstrap starting.
I20260807 06:35:48.728025 24540 tablet_bootstrap.cc:654] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:48.728386 24540 log.cc:824] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:48.729302 24540 tablet_bootstrap.cc:492] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: No bootstrap required, opened a new log
I20260807 06:35:48.729372 24540 ts_tablet_manager.cc:1397] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:35:48.732129 24540 raft_consensus.cc:348] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.732369 24540 raft_consensus.cc:374] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:48.732456 24540 raft_consensus.cc:729] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aa1f72e96064030a86840b7689da44d, State: Initialized, Role: FOLLOWER
I20260807 06:35:48.732667 24540 consensus_queue.cc:260] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.733016 24540 ts_tablet_manager.cc:1428] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 06:35:48.733208 24485 heartbeater.cc:503] Master 127.2.243.62:32811 was elected leader, sending a full tablet report...
I20260807 06:35:48.736693 24538 tablet_bootstrap.cc:492] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Bootstrap starting.
I20260807 06:35:48.738216 24538 tablet_bootstrap.cc:654] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:48.738787 24538 log.cc:824] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:48.739673 24538 tablet_bootstrap.cc:492] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: No bootstrap required, opened a new log
I20260807 06:35:48.739740 24538 ts_tablet_manager.cc:1397] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:35:48.742201 24538 raft_consensus.cc:348] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.742362 24538 raft_consensus.cc:374] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:48.742394 24538 raft_consensus.cc:729] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 758eead109c04bc5b2ad63c5e5f51681, State: Initialized, Role: FOLLOWER
I20260807 06:35:48.742659 24538 consensus_queue.cc:260] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.743022 24538 ts_tablet_manager.cc:1428] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:35:48.743153 24264 heartbeater.cc:503] Master 127.2.243.62:32811 was elected leader, sending a full tablet report...
W20260807 06:35:48.752424 24049 tablet.cc:2369] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:35:48.769730 24539 raft_consensus.cc:482] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:35:48.769883 24539 raft_consensus.cc:504] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.770292 24539 leader_election.cc:288] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 758eead109c04bc5b2ad63c5e5f51681 (127.2.243.2:41929), 2aa1f72e96064030a86840b7689da44d (127.2.243.3:39115)
I20260807 06:35:48.783783 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.771386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:57021 (local address 127.2.243.2:41929)
0807 06:35:48.771435 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:48.771437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:48.776774 (+ 5337us) server_negotiation.cc:407] Connection header received
0807 06:35:48.777329 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:48.777330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:48.777343 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:48.777372 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:48.779842 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.780584 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.781920 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.782492 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.782539 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.782550 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:48.783655 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0807 06:35:48.783702 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:35:48.784092 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.770524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:41929 (local address 127.2.243.1:57021)
0807 06:35:48.776676 (+ 6152us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:48.776693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:48.777250 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:48.777427 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:48.777428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:48.777429 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:48.777481 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.777482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.780684 (+ 3202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.780685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.781816 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.781819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.782629 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.782630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.782697 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.782701 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:35:48.783933 (+ 1232us) client_negotiation.cc:240] Negotiation successful
0807 06:35:48.783977 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6108,"thread_start_us":415,"threads_started":1}
I20260807 06:35:48.784431 24192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ed4c19107042d787194849495f4266" candidate_uuid: "428e31e9107f40ffb6bde5f500e7b933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "758eead109c04bc5b2ad63c5e5f51681" is_pre_election: true
I20260807 06:35:48.784626 24192 raft_consensus.cc:2454] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 428e31e9107f40ffb6bde5f500e7b933 in term 0.
I20260807 06:35:48.784874 23885 leader_election.cc:302] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 428e31e9107f40ffb6bde5f500e7b933, 758eead109c04bc5b2ad63c5e5f51681; no voters:
I20260807 06:35:48.785027 24539 raft_consensus.cc:2781] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:35:48.785076 24539 raft_consensus.cc:482] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:35:48.785105 24539 raft_consensus.cc:3037] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:48.785987 24550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.771063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:39115 (local address 127.2.243.1:58035)
0807 06:35:48.776824 (+ 5761us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:48.776832 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:48.776914 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:48.777119 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:48.777120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:48.777121 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:48.777199 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.777201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.778472 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.778473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.779725 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:48.779728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.783489 (+ 3761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.783490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:48.783569 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.783572 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:35:48.785852 (+ 2280us) client_negotiation.cc:240] Negotiation successful
0807 06:35:48.785901 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5747,"thread_start_us":69,"threads_started":1}
I20260807 06:35:48.786227 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.771296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:58035 (local address 127.2.243.3:39115)
0807 06:35:48.771388 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:48.771390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:48.776878 (+ 5488us) server_negotiation.cc:407] Connection header received
0807 06:35:48.776990 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:48.776992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:48.777015 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:48.777047 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:48.777556 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.778368 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.782787 (+ 4419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.783373 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.783420 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.783429 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:48.786111 (+ 2682us) server_negotiation.cc:299] Negotiation successful
0807 06:35:48.786164 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:35:48.786012 24539 raft_consensus.cc:504] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.786451 24539 leader_election.cc:288] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [CANDIDATE]: Term 1 election: Requested vote from peers 758eead109c04bc5b2ad63c5e5f51681 (127.2.243.2:41929), 2aa1f72e96064030a86840b7689da44d (127.2.243.3:39115)
I20260807 06:35:48.786504 24411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ed4c19107042d787194849495f4266" candidate_uuid: "428e31e9107f40ffb6bde5f500e7b933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aa1f72e96064030a86840b7689da44d" is_pre_election: true
I20260807 06:35:48.786681 24410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ed4c19107042d787194849495f4266" candidate_uuid: "428e31e9107f40ffb6bde5f500e7b933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aa1f72e96064030a86840b7689da44d"
I20260807 06:35:48.786715 24192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ed4c19107042d787194849495f4266" candidate_uuid: "428e31e9107f40ffb6bde5f500e7b933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "758eead109c04bc5b2ad63c5e5f51681"
I20260807 06:35:48.786806 24192 raft_consensus.cc:3037] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:48.786685 24411 raft_consensus.cc:2454] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 428e31e9107f40ffb6bde5f500e7b933 in term 0.
I20260807 06:35:48.787719 24192 raft_consensus.cc:2454] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 428e31e9107f40ffb6bde5f500e7b933 in term 1.
I20260807 06:35:48.787909 23885 leader_election.cc:302] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 428e31e9107f40ffb6bde5f500e7b933, 758eead109c04bc5b2ad63c5e5f51681; no voters: 2aa1f72e96064030a86840b7689da44d
I20260807 06:35:48.788070 24539 raft_consensus.cc:2781] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:48.788242 24539 raft_consensus.cc:686] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [term 1 LEADER]: Becoming Leader. State: Replica: 428e31e9107f40ffb6bde5f500e7b933, State: Running, Role: LEADER
I20260807 06:35:48.788322 24539 consensus_queue.cc:237] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } }
I20260807 06:35:48.789191 23735 catalog_manager.cc:5495] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e31e9107f40ffb6bde5f500e7b933 (127.2.243.1). New cstate: current_term: 1 leader_uuid: "428e31e9107f40ffb6bde5f500e7b933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "428e31e9107f40ffb6bde5f500e7b933" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 39345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 } health_report { overall_health: UNKNOWN } } }
I20260807 06:35:48.796387 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.793325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.2.243.1:39345)
0807 06:35:48.793370 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:48.793372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:48.793382 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:35:48.793456 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:48.793456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:48.793477 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:48.793545 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:35:48.793717 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.794457 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.795483 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:48.795805 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:48.795870 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:48.796044 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:35:48.796169 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:35:48.796248 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:48.796293 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:35:48.796313 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:35:48.799162 24411 raft_consensus.cc:3037] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:48.799365 24192 raft_consensus.cc:1261] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 1 FOLLOWER]: Refusing update from remote peer 428e31e9107f40ffb6bde5f500e7b933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:48.799657 24539 consensus_queue.cc:1035] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "758eead109c04bc5b2ad63c5e5f51681" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:48.800172 24411 raft_consensus.cc:1261] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 1 FOLLOWER]: Refusing update from remote peer 428e31e9107f40ffb6bde5f500e7b933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:48.801139 24539 consensus_queue.cc:1035] T 92ed4c19107042d787194849495f4266 P 428e31e9107f40ffb6bde5f500e7b933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2aa1f72e96064030a86840b7689da44d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 39115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:48.803766 24557 mvcc.cc:205] Tried to move back new op lower bound from 7315802311878955008 to 7315802311837548544. Current Snapshot: MvccSnapshot[applied={T|T < 7315802311878955008}]
I20260807 06:35:48.804641 24563 mvcc.cc:205] Tried to move back new op lower bound from 7315802311878955008 to 7315802311837548544. Current Snapshot: MvccSnapshot[applied={T|T < 7315802311878955008}]
I20260807 06:35:48.806187 24558 mvcc.cc:205] Tried to move back new op lower bound from 7315802311878955008 to 7315802311837548544. Current Snapshot: MvccSnapshot[applied={T|T < 7315802311878955008}]
I20260807 06:35:48.808080 23734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:48.808177 23734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:48.808930 23734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2064d76940bf4986b910a8d76897da4b: Sending DeleteTablet for 3 replicas of tablet 92ed4c19107042d787194849495f4266
I20260807 06:35:48.809393 23932 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ed4c19107042d787194849495f4266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:48 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:38156
I20260807 06:35:48.809473 24165 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ed4c19107042d787194849495f4266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:48 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:46240
I20260807 06:35:48.809675 24374 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ed4c19107042d787194849495f4266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:48 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:60422
I20260807 06:35:48.810259 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 23803
I20260807 06:35:48.820586 24579 tablet_replica.cc:330] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: stopping tablet replica
I20260807 06:35:48.820734 24579 raft_consensus.cc:2229] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:48.820865 24579 raft_consensus.cc:2258] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:48.821075 24580 tablet_replica.cc:330] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: stopping tablet replica
I20260807 06:35:48.821192 24580 raft_consensus.cc:2229] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:48.821305 24580 raft_consensus.cc:2258] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:48.821822 24580 ts_tablet_manager.cc:1905] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:35:48.823755 24580 ts_tablet_manager.cc:1918] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:48.823858 24580 log.cc:1196] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/92ed4c19107042d787194849495f4266
I20260807 06:35:48.824163 24580 ts_tablet_manager.cc:1939] T 92ed4c19107042d787194849495f4266 P 2aa1f72e96064030a86840b7689da44d: Deleting consensus metadata
I20260807 06:35:48.824839 24579 ts_tablet_manager.cc:1905] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:35:48.825961 23711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:39345 (error 108)
I20260807 06:35:48.826567 24579 ts_tablet_manager.cc:1918] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:48.826661 24579 log.cc:1196] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/92ed4c19107042d787194849495f4266
I20260807 06:35:48.826933 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 24052
I20260807 06:35:48.826951 24579 ts_tablet_manager.cc:1939] T 92ed4c19107042d787194849495f4266 P 758eead109c04bc5b2ad63c5e5f51681: Deleting consensus metadata
I20260807 06:35:48.826941 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.826798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:39345 (local address 127.0.0.1:38178)
0807 06:35:48.826849 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:48.826901 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:39345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 06:35:48.844525 23711 catalog_manager.cc:4584] TS 428e31e9107f40ffb6bde5f500e7b933 (127.2.243.1:39345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92ed4c19107042d787194849495f4266: Network error: Client connection negotiation failed: client connection to 127.2.243.1:39345: connect: Connection refused (error 111)
I20260807 06:35:48.844731 23711 catalog_manager.cc:4841] TS 2aa1f72e96064030a86840b7689da44d (127.2.243.3:39115): tablet 92ed4c19107042d787194849495f4266 (table test-table [id=c5441c23fae449eeab79fc36fd0d9733]) successfully deleted
I20260807 06:35:48.858681 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 24269
I20260807 06:35:48.858729 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:48.858594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:41929 (local address 127.0.0.1:46250)
0807 06:35:48.858658 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:48.858687 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.2:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 06:35:48.868104 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 23684
I20260807 06:35:48.888407 3020 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 06:35:49.233062 3020 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 06:35:49.233156 3020 mini_ranger.cc:66] Stopping Ranger...
I20260807 06:35:49.577941 3020 mini_ranger.cc:68] Stopped Ranger
2026-08-07 06:35:49.578 UTC [22211] LOG: received smart shutdown request
2026-08-07 06:35:49.582 UTC [22211] LOG: background worker "logical replication launcher" (PID 22224) exited with exit code 1
2026-08-07 06:35:49.583 UTC [22218] LOG: shutting down
2026-08-07 06:35:49.746 UTC [22211] LOG: database system is shut down
2026-08-07T06:35:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48004 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:35:49 dist-test-slave-rwrg krb5kdc[24597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:35:49 dist-test-slave-rwrg krb5kdc[24597](info): set up 2 sockets
Aug 07 06:35:49 dist-test-slave-rwrg krb5kdc[24597](info): commencing operation
krb5kdc: starting...
W20260807 06:35:51.118902 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:35:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:35:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:35:51.241473 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33123
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:33123,127.2.243.61:36389,127.2.243.60:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:51.321275 24614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:51.321522 24614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:51.321557 24614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:51.322685 24614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:51.322758 24614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:51.322782 24614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:51.322803 24614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:51.322831 24614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:51.324291 24614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:33123,127.2.243.61:36389,127.2.243.60:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:33123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:51.324594 24614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:51.324822 24614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:51.329734 24620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:51.329734 24622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:51.329768 24626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:51.330009 24614 server_base.cc:1034] running on GCE node
I20260807 06:35:51.330442 24614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:51.330782 24614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:51.331936 24614 hybrid_clock.cc:648] HybridClock initialized: now 1786084551331921 us; error 39 us; skew 500 ppm
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:51.334625 24614 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:35:51.335124 24614 webserver.cc:460] Webserver started at http://127.2.243.62:37601/ using document root <none> and password file <none>
I20260807 06:35:51.335377 24614 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:51.335422 24614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:51.335593 24614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:51.336790 24614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "d26abafb13144c4f8995a94cad0f4358"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "d2762d015778d4554021ee552c71f832"
server_key_iv: "5f40bf1cf9a5e35a79726e9592179060"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.337164 24614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "d26abafb13144c4f8995a94cad0f4358"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "d2762d015778d4554021ee552c71f832"
server_key_iv: "5f40bf1cf9a5e35a79726e9592179060"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.338594 24614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:51.339365 24638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:51.339560 24614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:51.339653 24614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "d26abafb13144c4f8995a94cad0f4358"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "d2762d015778d4554021ee552c71f832"
server_key_iv: "5f40bf1cf9a5e35a79726e9592179060"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.339726 24614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:51.349032 24614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:51.350924 24614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:51.351085 24614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:51.355425 24614 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:33123
I20260807 06:35:51.355459 24724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:33123 every 8 connection(s)
I20260807 06:35:51.355823 24614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:35:51.356211 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 24614
I20260807 06:35:51.356325 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:35:51.356602 3020 external_mini_cluster.cc:1428] Setting key f85c072b7d52fe7f6a0bc47f065bd218
I20260807 06:35:51.356873 24725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:51.358640 24725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.243.61@KRBTEST.COM; defaulting to no policy
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.61@KRBTEST.COM" created.
I20260807 06:35:51.369844 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.359491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33123 (local address 127.0.0.1:53640)
0807 06:35:51.359665 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.359718 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.360263 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.360608 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.360611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.360884 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.361030 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.361034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.362697 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.362697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.363346 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.363348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.364083 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.364084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.364161 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.364744 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.364752 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.366224 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.368185 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.368192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.368193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.368377 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.368761 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.368762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.368762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.368875 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.369391 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.369397 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.369456 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.369532 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.369669 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260807 06:35:51.369957 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.359730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.2.243.62:33123)
0807 06:35:51.359885 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.359889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.359923 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:35:51.360423 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.360426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.360445 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.360602 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.361189 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.362581 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.363437 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.363948 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.364096 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.366344 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.366352 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.366352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.366359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.368080 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.368576 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.368580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.368580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.368676 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.368946 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.368946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.368947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.369308 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.369414 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.369614 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.369765 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260807 06:35:51.370927 24725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:35:51.371337 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.371227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40618)
0807 06:35:51.371276 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.371318 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:35:51.371515 24639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
I20260807 06:35:51.371949 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.371885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40632)
0807 06:35:51.371911 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.371934 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260807 06:35:51.372076 24725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:36389: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab.
Entry for principal kudu/127.2.243.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab.
I20260807 06:35:51.385922 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:36389
--webserver_interface=127.2.243.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab
--principal=kudu/127.2.243.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:33123,127.2.243.61:36389,127.2.243.60:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:35:51.421320 24725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } attempt: 1
I20260807 06:35:51.421880 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.421730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40642)
0807 06:35:51.421803 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.421848 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 06:35:51.422279 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.422187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40648)
0807 06:35:51.422236 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.422262 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:35:51.422421 24725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:36389: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
W20260807 06:35:51.469604 24735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:51.469808 24735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:51.469830 24735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:51.471040 24735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:51.471084 24735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:51.471099 24735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:51.471115 24735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:51.471127 24735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:51.472663 24735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:33123,127.2.243.61:36389,127.2.243.60:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab
--principal=kudu/127.2.243.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.243.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:51.472910 24735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:51.473157 24735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:51.478009 24747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:51.478137 24735 server_base.cc:1034] running on GCE node
W20260807 06:35:51.478010 24749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:51.478010 24746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:51.478526 24735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:51.478814 24735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:51.479954 24735 hybrid_clock.cc:648] HybridClock initialized: now 1786084551479934 us; error 37 us; skew 500 ppm
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:51.485102 24735 init.cc:377] Logged in from keytab as kudu/127.2.243.61@KRBTEST.COM (short username kudu)
I20260807 06:35:51.485549 24735 webserver.cc:460] Webserver started at http://127.2.243.61:32961/ using document root <none> and password file <none>
I20260807 06:35:51.485734 24735 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:51.485773 24735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:51.485860 24735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:51.486872 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data/instance:
uuid: "3bc46c0c3c3445cb90602e8210ab1bb5"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "9ea0a284bc6d273e16be7907b6cf6011"
server_key_iv: "8ea7bbd97506f9c76f1922c012eb0784"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.487156 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/wal/instance:
uuid: "3bc46c0c3c3445cb90602e8210ab1bb5"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "9ea0a284bc6d273e16be7907b6cf6011"
server_key_iv: "8ea7bbd97506f9c76f1922c012eb0784"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.488404 24735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 06:35:51.489251 24759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:51.489444 24735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:51.489552 24735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/wal
uuid: "3bc46c0c3c3445cb90602e8210ab1bb5"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "9ea0a284bc6d273e16be7907b6cf6011"
server_key_iv: "8ea7bbd97506f9c76f1922c012eb0784"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.489652 24735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:51.490576 24725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } attempt: 2
I20260807 06:35:51.490953 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.490834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40664)
0807 06:35:51.490891 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.490928 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:35:51.491369 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.491277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40670)
0807 06:35:51.491323 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.491346 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 06:35:51.491467 24725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:36389: Network error: Client connection negotiation failed: client connection to 127.2.243.61:36389: connect: Connection refused (error 111)
I20260807 06:35:51.511029 24735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:51.513487 24735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:51.513662 24735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:51.517817 24735 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.61:36389
I20260807 06:35:51.517860 24832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.61:36389 every 8 connection(s)
I20260807 06:35:51.518210 24735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
I20260807 06:35:51.519047 24833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:51.520615 24833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:35:51.521548 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 24735
I20260807 06:35:51.521677 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-1/wal/instance
I20260807 06:35:51.521863 3020 external_mini_cluster.cc:1428] Setting key b48a88ae96470d143c94532d9ce54a3b
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.243.60@KRBTEST.COM; defaulting to no policy
I20260807 06:35:51.531167 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.521326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.2.243.62:33123)
0807 06:35:51.521376 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.521377 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.521609 (+ 232us) server_negotiation.cc:407] Connection header received
0807 06:35:51.522161 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.522163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.522180 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.522253 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.522993 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.523688 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.525090 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.525285 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.525415 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.528041 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.528042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.528043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.528051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.529760 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.530158 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.530159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.530160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.530237 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.530478 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.530479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.530480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.530617 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.530689 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.530950 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.531040 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:35:51.531229 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.521279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33123 (local address 127.0.0.1:53644)
0807 06:35:51.521441 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.521481 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.522061 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.522320 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.522323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.522625 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.522907 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.522912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.523781 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.523782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.524989 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.524991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.525425 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.525426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.525569 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.526212 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.526220 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.527940 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.529844 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.529851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.529853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.530074 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.530313 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.530314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.530314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.530407 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.530684 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.530688 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.530794 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.530918 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.531054 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":109,"threads_started":1}
I20260807 06:35:51.531797 24833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.60@KRBTEST.COM" created.
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:35:51.539975 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.532097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40672)
0807 06:35:51.532147 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.532158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.532199 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.532477 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.532477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.532659 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.532707 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.532708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.533639 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.533640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.534338 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.534341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.534816 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.534817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.534898 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.535337 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.535344 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.536764 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.538888 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.538889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.538889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.539110 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.539322 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.539323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.539323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.539377 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.539673 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.539673 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.539710 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.539774 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.539870 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 06:35:51.540074 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.532168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.2.243.61:36389)
0807 06:35:51.532338 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.532342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.532357 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:35:51.532373 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.532374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.532384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.532462 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.532788 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.533539 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.534427 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.534737 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.534797 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.536875 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.536883 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.536883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.536889 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.538797 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.539192 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.539195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.539196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.539255 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.539445 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.539446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.539447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.539593 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.539680 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.539813 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.539940 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260807 06:35:51.540614 24833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:35:51.540946 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.540848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36750)
0807 06:35:51.540887 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.540927 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 06:35:51.541111 24761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
I20260807 06:35:51.541535 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.541447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36762)
0807 06:35:51.541496 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.541519 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 06:35:51.541646 24833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:43273: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab.
Entry for principal kudu/127.2.243.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab.
I20260807 06:35:51.550988 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:43273
--webserver_interface=127.2.243.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab
--principal=kudu/127.2.243.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:33123,127.2.243.61:36389,127.2.243.60:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:35:51.582628 24725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } attempt: 3
I20260807 06:35:51.590847 24833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } attempt: 1
I20260807 06:35:51.591266 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.591125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36772)
0807 06:35:51.591199 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.591237 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 06:35:51.591629 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.591511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36780)
0807 06:35:51.591557 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.591610 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:35:51.591727 24833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:43273: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:35:51.596792 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.583107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.2.243.61:36389)
0807 06:35:51.583177 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.583184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.583196 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:35:51.583277 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.583278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.583292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.583366 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.583728 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.584223 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.585052 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.585267 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.585343 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.594307 (+ 8964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.594309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.594309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.594316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.595714 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.596095 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.596095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.596096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.596146 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.596343 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.596344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.596344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.596444 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.596532 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.596675 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.596737 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:35:51.596858 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.583034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40682)
0807 06:35:51.583091 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.583104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.583187 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.583372 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.583373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.583548 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.583640 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.583642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.584326 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.584328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.584957 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.584960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.585340 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.585341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.585423 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.592740 (+ 7317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.592745 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.594222 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.595800 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.595801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.595802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.596012 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.596209 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.596210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.596210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.596280 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.596522 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.596523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.596582 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.596658 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.596756 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 06:35:51.597288 24725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:35:51.597651 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.597545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36784)
0807 06:35:51.597593 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.597626 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 06:35:51.597963 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.597872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36786)
0807 06:35:51.597921 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.597947 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:35:51.598067 24725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:43273: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
I20260807 06:35:51.629199 24725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } attempt: 1
I20260807 06:35:51.629673 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.629510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36802)
0807 06:35:51.629589 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.629634 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 06:35:51.630066 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.629960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36816)
0807 06:35:51.630009 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.630040 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:35:51.630182 24725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:43273: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
W20260807 06:35:51.636636 24844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:51.636879 24844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:51.636919 24844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:51.638327 24844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:51.638381 24844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:51.638401 24844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:51.638419 24844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:51.638435 24844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:51.640352 24844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:33123,127.2.243.61:36389,127.2.243.60:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab
--principal=kudu/127.2.243.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.243.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:51.640679 24844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:51.640919 24844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:51.647464 24858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:51.647548 24855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:51.647464 24861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:51.649179 24844 server_base.cc:1034] running on GCE node
I20260807 06:35:51.649510 24844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:51.649883 24844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:51.651016 24844 hybrid_clock.cc:648] HybridClock initialized: now 1786084551650997 us; error 33 us; skew 500 ppm
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:51.653934 24844 init.cc:377] Logged in from keytab as kudu/127.2.243.60@KRBTEST.COM (short username kudu)
I20260807 06:35:51.654467 24844 webserver.cc:460] Webserver started at http://127.2.243.60:37781/ using document root <none> and password file <none>
I20260807 06:35:51.654732 24844 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:51.654804 24844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:51.654961 24844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:51.656538 24844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data/instance:
uuid: "d84e715d02374f91bcf2ace48eb1ed0b"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "5396227e02208451f0d2fec1ef97953a"
server_key_iv: "eaf8e65a1aa98a0076344c2446ff82ff"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.656996 24844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/wal/instance:
uuid: "d84e715d02374f91bcf2ace48eb1ed0b"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "5396227e02208451f0d2fec1ef97953a"
server_key_iv: "eaf8e65a1aa98a0076344c2446ff82ff"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.659919 24833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } attempt: 2
I20260807 06:35:51.660405 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.660266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36820)
0807 06:35:51.660335 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.660370 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:35:51.660441 24844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 06:35:51.660772 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.660667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36832)
0807 06:35:51.660720 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.660749 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:35:51.660851 24833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:43273: Network error: Client connection negotiation failed: client connection to 127.2.243.60:43273: connect: Connection refused (error 111)
I20260807 06:35:51.661789 24875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:51.661940 24844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:51.662037 24844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/wal
uuid: "d84e715d02374f91bcf2ace48eb1ed0b"
format_stamp: "Formatted at 2026-08-07 06:35:51 on dist-test-slave-rwrg"
server_key: "5396227e02208451f0d2fec1ef97953a"
server_key_iv: "eaf8e65a1aa98a0076344c2446ff82ff"
server_key_version: "encryptionkey@0"
I20260807 06:35:51.662125 24844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:51.681000 24844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:51.683768 24844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:51.683912 24844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:51.688735 24844 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.60:43273
I20260807 06:35:51.688771 24965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.60:43273 every 8 connection(s)
I20260807 06:35:51.689144 24844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
I20260807 06:35:51.690047 24966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:51.691606 24966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:35:51.696998 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 24844
I20260807 06:35:51.697082 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786084392481039-3020-0/minicluster-data/master-2/wal/instance
I20260807 06:35:51.697271 3020 external_mini_cluster.cc:1428] Setting key 79bc0854280aae7bdaf8d4ebc5bdbf10
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:51.701256 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.692346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.2.243.62:33123)
0807 06:35:51.692400 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.692402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.692658 (+ 256us) server_negotiation.cc:407] Connection header received
0807 06:35:51.693191 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.693192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.693204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.693266 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.693965 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.694428 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.695370 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.695590 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.695667 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.698017 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.698018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.698019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.698027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.699732 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.700139 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.700140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.700141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.700206 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.700482 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.700483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.700483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.700676 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.700710 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.701067 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.701146 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:35:51.701371 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.692289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:33123 (local address 127.0.0.1:53656)
0807 06:35:51.692469 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.692547 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.693076 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.693332 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.693336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.693688 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.693874 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.693878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.694518 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.694520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.695278 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.695280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.695674 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.695676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.695808 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.696252 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.696260 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.697913 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.699812 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.699818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.699820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.700045 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.700284 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.700284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.700285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.700414 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.700771 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.700777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.700920 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.701026 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.701163 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260807 06:35:51.701932 24966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:35:51.705361 24725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } attempt: 2
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:35:51.707899 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.697922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.2.243.60:43273)
0807 06:35:51.698070 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.698075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.698096 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:35:51.698117 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.698119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.698133 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.698228 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.699039 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.700114 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.701039 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.701496 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.701574 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.704043 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.704051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.704051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.704058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.706645 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.706992 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.706994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.706995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.707081 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.707288 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.707289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.707289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.707449 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.707641 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.707720 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.707797 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:35:51.710860 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.702233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:36389 (local address 127.0.0.1:40686)
0807 06:35:51.702364 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.702375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.702440 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.702558 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.702559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.702706 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.702761 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.702763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.703334 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.703334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.703872 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.703873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.704354 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.704355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.704432 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.704739 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.704744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.706301 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.709606 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.709606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.709607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.709868 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.710133 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.710134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.710134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.710218 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.710539 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.710540 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.710604 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.710670 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.710760 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 06:35:51.710978 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.702311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.2.243.61:36389)
0807 06:35:51.702377 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.702379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.702426 (+ 47us) server_negotiation.cc:407] Connection header received
0807 06:35:51.702499 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.702500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.702509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.702551 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.702838 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.703252 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.703957 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.704284 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.704356 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.706396 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.706397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.706398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.706405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.709535 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.709954 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.709955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.709956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.710056 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.710278 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.710278 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.710279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.710459 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.710546 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.710714 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.710816 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:35:51.711627 24966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:35:51.715297 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.705989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36858)
0807 06:35:51.706026 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.706037 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.706076 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.706320 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.706321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.706485 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.706539 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.706540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.707381 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.707382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.708109 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.708112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.708611 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.708612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.708673 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.708992 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.708997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.710637 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.714232 (+ 3595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.714233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.714233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.714449 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.714691 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.714691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.714692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.714753 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.715030 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.715030 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.715077 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.715125 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.715209 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 06:35:51.715794 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.706038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36858 (local address 127.2.243.60:43273)
0807 06:35:51.706205 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.706207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.706213 (+ 6us) server_negotiation.cc:407] Connection header received
0807 06:35:51.706230 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.706231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.706242 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.706309 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.706619 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.707310 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.708179 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.708500 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.708606 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.710738 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.710740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.710740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.710747 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.714129 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.714548 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.714549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.714549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.714611 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.714829 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.714830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.714830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.714953 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.715533 (+ 580us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.715574 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.715690 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:35:51.717857 24725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: Bootstrap starting.
I20260807 06:35:51.718730 24725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:51.719000 24725 log.cc:824] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:51.719861 24725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: No bootstrap required, opened a new log
I20260807 06:35:51.720856 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.711042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.2.243.62:33123)
0807 06:35:51.711122 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.711124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.711132 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:35:51.711151 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.711152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.711161 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.711216 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.711977 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.712790 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.713717 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.714006 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.714056 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.717498 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.717499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.717499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.717506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.719293 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.719682 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.719683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.719684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.719740 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.719951 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.719952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.719952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.720116 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.720635 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.720680 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.720756 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:35:51.721355 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.711880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36870)
0807 06:35:51.711918 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.711926 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.711959 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.712114 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.712115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.712242 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.712265 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.712266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.713505 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.713506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.714207 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.714209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.714666 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.714667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.714714 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.715086 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.715092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.717943 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.720378 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.720378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.720379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.720613 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.720796 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.720797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.720797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.720842 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.721124 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.721125 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.721159 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.721219 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.721301 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 06:35:51.721624 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.711914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.2.243.60:43273)
0807 06:35:51.711959 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.711961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.711970 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:35:51.712031 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.712032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.712064 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.712111 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.712349 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.713422 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.714287 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.714567 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.714624 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.718029 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.718029 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.718030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.718035 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.720306 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.720687 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.720688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.720688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.720737 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.720897 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.720898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.720898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.721045 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.721105 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.721456 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.721527 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:35:51.722318 24725 raft_consensus.cc:348] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.722470 24725 raft_consensus.cc:374] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:51.722504 24725 raft_consensus.cc:729] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26abafb13144c4f8995a94cad0f4358, State: Initialized, Role: FOLLOWER
I20260807 06:35:51.722646 24725 consensus_queue.cc:260] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.722929 24725 sys_catalog.cc:564] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:51.723083 24966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b: Bootstrap starting.
I20260807 06:35:51.723150 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.723220 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:35:51.723748 24966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:51.724063 24966 log.cc:824] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b: Log is configured to *not* fsync() on all Append() calls
W20260807 06:35:51.725049 24997 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:35:51.725106 24997 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:35:51.725719 24966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b: No bootstrap required, opened a new log
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:35:51.727818 24966 raft_consensus.cc:348] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.727950 24966 raft_consensus.cc:374] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:51.727982 24966 raft_consensus.cc:729] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84e715d02374f91bcf2ace48eb1ed0b, State: Initialized, Role: FOLLOWER
I20260807 06:35:51.728097 24966 consensus_queue.cc:260] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.728385 24966 sys_catalog.cc:564] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:51.728354 25005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.728442 25005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:35:51.726333 24614 master_runner.cc:186] Error getting master registration for 127.2.243.61:36389: OK, instance_id { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" instance_seqno: 1786084551516573 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:35:51.729779 24614 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 06:35:51.730535 25026 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:35:51.730628 25026 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:35:51.732952 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.721534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.2.243.61:36389)
0807 06:35:51.721646 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.721648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.721656 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:35:51.721678 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.721679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.721688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.721743 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.722325 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.722993 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.723919 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.724230 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.724308 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.726943 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.726944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.726944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.726951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.731362 (+ 4411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.731749 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.731750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.731751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.731819 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.732031 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.732032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.732032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.732148 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.732802 (+ 654us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.732835 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.732893 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 06:35:51.731877 24844 master_runner.cc:186] Error getting master registration for 127.2.243.61:36389: OK, instance_id { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" instance_seqno: 1786084551516573 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:35:51.734140 24844 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:35:51.734184 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:35:51.743727 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.735520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53672 (local address 127.2.243.62:33123)
0807 06:35:51.735560 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.735562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.735869 (+ 307us) server_negotiation.cc:407] Connection header received
0807 06:35:51.735950 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.735950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.735961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.736014 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.736850 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.737508 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.738498 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.738788 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.738825 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.740242 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.740244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.740244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.740251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.742344 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.742813 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.742814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.742814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.742872 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.743078 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.743079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.743080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.743217 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.743288 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.743553 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.743629 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:35:51.744966 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.735610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.2.243.60:43273)
0807 06:35:51.735670 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.735671 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.735938 (+ 267us) server_negotiation.cc:407] Connection header received
0807 06:35:51.736026 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.736027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.736038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.736103 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.736635 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.737303 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.738261 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.738523 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.738565 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.740167 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.740168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.740169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.740176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.742825 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.743219 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.743220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.743221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.743285 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.743511 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.743512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.743512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.743634 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.743713 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.744799 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.744880 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:35:51.747682 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.735822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.2.243.61:36389)
0807 06:35:51.735856 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.735859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.740664 (+ 4805us) server_negotiation.cc:407] Connection header received
0807 06:35:51.740726 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.740727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.740737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.740800 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.741305 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.742048 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.743023 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.743344 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.743385 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.744851 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.744852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.744853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.744859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.746454 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.746849 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.746850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.746851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.746913 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.747129 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.747130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.747130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.747262 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.747358 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.747505 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.747587 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:35:51.751979 24833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } attempt: 3
Aug 07 06:35:51 dist-test-slave-rwrg krb5kdc[24597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084551, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:35:51.760036 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.752484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.2.243.60:43273)
0807 06:35:51.752561 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:51.752563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:51.752570 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:35:51.752588 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:51.752589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:51.752598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:51.752669 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:51.752968 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.753674 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.754626 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.754943 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.755014 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.757131 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:51.757132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:51.757133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:51.757140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:51.758867 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.759241 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.759242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.759242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.759301 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:51.759507 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:51.759507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:51.759508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:51.759636 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:51.759703 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:51.759861 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 06:35:51.759934 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:35:51.760038 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:51.752253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:43273 (local address 127.0.0.1:36894)
0807 06:35:51.752351 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:51.752361 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:51.752398 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:51.752718 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:51.752719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:51.752839 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:51.752871 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.752872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.753763 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.753764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.754540 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:51.754542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:51.755028 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:51.755029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:51.755087 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:51.755529 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:51.755535 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:51.757043 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:51.758948 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.758949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.758950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.759162 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.759371 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:51.759372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:51.759372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:51.759441 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:51.759706 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:51.759707 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:51.759752 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:35:51.759826 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:35:51.759937 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 06:35:51.762533 24833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: Bootstrap starting.
I20260807 06:35:51.763355 24833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:51.763685 24833 log.cc:824] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:51.764755 24833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: No bootstrap required, opened a new log
I20260807 06:35:51.766938 24833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.767092 24833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:51.767127 24833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc46c0c3c3445cb90602e8210ab1bb5, State: Initialized, Role: FOLLOWER
I20260807 06:35:51.767279 24833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.767607 24833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:51.767565 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.767755 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:35:51.770524 25055 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:35:51.770627 25055 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:35:51.771374 24735 master_runner.cc:263] No leader master found from master 3bc46c0c3c3445cb90602e8210ab1bb5
I20260807 06:35:51.771428 24735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:35:51.780623 25005 raft_consensus.cc:482] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:35:51.780731 25005 raft_consensus.cc:504] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.780973 25005 leader_election.cc:288] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d26abafb13144c4f8995a94cad0f4358 (127.2.243.62:33123), 3bc46c0c3c3445cb90602e8210ab1bb5 (127.2.243.61:36389)
I20260807 06:35:51.781421 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" is_pre_election: true
I20260807 06:35:51.781562 24796 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d84e715d02374f91bcf2ace48eb1ed0b in term 0.
I20260807 06:35:51.781934 24683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26abafb13144c4f8995a94cad0f4358" is_pre_election: true
I20260807 06:35:51.782094 24683 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d84e715d02374f91bcf2ace48eb1ed0b in term 0.
I20260807 06:35:51.782394 24876 leader_election.cc:302] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc46c0c3c3445cb90602e8210ab1bb5, d84e715d02374f91bcf2ace48eb1ed0b; no voters:
I20260807 06:35:51.782615 25005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:35:51.782668 25005 raft_consensus.cc:482] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:35:51.782691 25005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:51.783584 25005 raft_consensus.cc:504] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.783715 25005 leader_election.cc:288] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [CANDIDATE]: Term 1 election: Requested vote from peers d26abafb13144c4f8995a94cad0f4358 (127.2.243.62:33123), 3bc46c0c3c3445cb90602e8210ab1bb5 (127.2.243.61:36389)
I20260807 06:35:51.784087 24683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26abafb13144c4f8995a94cad0f4358"
I20260807 06:35:51.784170 24683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:51.785090 24683 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d84e715d02374f91bcf2ace48eb1ed0b in term 1.
I20260807 06:35:51.785324 24881 leader_election.cc:302] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d26abafb13144c4f8995a94cad0f4358, d84e715d02374f91bcf2ace48eb1ed0b; no voters:
I20260807 06:35:51.785454 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5"
I20260807 06:35:51.785526 24796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:51.786808 24796 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d84e715d02374f91bcf2ace48eb1ed0b in term 1.
I20260807 06:35:51.787118 25005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:51.787288 25005 raft_consensus.cc:686] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [term 1 LEADER]: Becoming Leader. State: Replica: d84e715d02374f91bcf2ace48eb1ed0b, State: Running, Role: LEADER
I20260807 06:35:51.787371 25005 consensus_queue.cc:237] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } }
I20260807 06:35:51.787742 25005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84e715d02374f91bcf2ace48eb1ed0b. Latest consensus state: current_term: 1 leader_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.787807 25005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:51.789567 25061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:35:51.789824 25061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:35:51.791399 24796 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [term 1 FOLLOWER]: Refusing update from remote peer d84e715d02374f91bcf2ace48eb1ed0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:51.791828 25058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:51.793385 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84e715d02374f91bcf2ace48eb1ed0b. Latest consensus state: current_term: 1 leader_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.793470 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:35:51.793937 24683 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [term 1 FOLLOWER]: Refusing update from remote peer d84e715d02374f91bcf2ace48eb1ed0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:51.794267 25005 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:51.795483 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84e715d02374f91bcf2ace48eb1ed0b. Latest consensus state: current_term: 1 leader_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.795572 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:35:51.796938 25058 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.797039 25058 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:51.797605 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.797683 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:35:51.797923 25064 mvcc.cc:205] Tried to move back new op lower bound from 7315802324135256064 to 7315802324121726976. Current Snapshot: MvccSnapshot[applied={T|T < 7315802324135256064}]
I20260807 06:35:51.798072 25065 mvcc.cc:205] Tried to move back new op lower bound from 7315802324135256064 to 7315802324121726976. Current Snapshot: MvccSnapshot[applied={T|T < 7315802324135256064}]
I20260807 06:35:51.799199 25005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.799273 25005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:51.799714 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26abafb13144c4f8995a94cad0f4358" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 33123 } } peers { permanent_uuid: "3bc46c0c3c3445cb90602e8210ab1bb5" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 36389 } } peers { permanent_uuid: "d84e715d02374f91bcf2ace48eb1ed0b" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 43273 } } }
I20260807 06:35:51.799788 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:35:51.800057 25067 mvcc.cc:205] Tried to move back new op lower bound from 7315802324135256064 to 7315802324121726976. Current Snapshot: MvccSnapshot[applied={T|T < 7315802324135256064}]
I20260807 06:35:51.801270 25061 catalog_manager.cc:1347] Generated new cluster ID: e08864cbed0644c285dc5058089e026a
I20260807 06:35:51.801318 25061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:35:51.835700 25061 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:35:51.836547 25061 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:35:51.849396 25061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d84e715d02374f91bcf2ace48eb1ed0b: Generated new TSK 0
I20260807 06:35:51.849720 25061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:35:52.725592 24997 catalog_manager.cc:1259] Loaded cluster ID: e08864cbed0644c285dc5058089e026a
I20260807 06:35:52.725669 24997 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: loading cluster ID for follower catalog manager: success
I20260807 06:35:52.726460 24997 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: acquiring CA information for follower catalog manager: success
I20260807 06:35:52.726758 24997 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d26abafb13144c4f8995a94cad0f4358: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:35:52.771133 25055 catalog_manager.cc:1259] Loaded cluster ID: e08864cbed0644c285dc5058089e026a
I20260807 06:35:52.771219 25055 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: loading cluster ID for follower catalog manager: success
I20260807 06:35:52.772076 25055 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: acquiring CA information for follower catalog manager: success
I20260807 06:35:52.772406 25055 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3bc46c0c3c3445cb90602e8210ab1bb5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:35:52.901371 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 24614
I20260807 06:35:52.911849 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 24735
I20260807 06:35:52.923570 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 24844
2026-08-07T06:35:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3063 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:35:52 dist-test-slave-rwrg krb5kdc[25086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:35:52 dist-test-slave-rwrg krb5kdc[25086](info): set up 2 sockets
Aug 07 06:35:52 dist-test-slave-rwrg krb5kdc[25086](info): commencing operation
krb5kdc: starting...
W20260807 06:35:54.178485 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:35:54 dist-test-slave-rwrg krb5kdc[25086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:35:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:35:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:35:54.303329 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:41873
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:41873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:54.385277 25108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:54.385543 25108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:54.385582 25108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:54.386711 25108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:54.386785 25108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:54.386809 25108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:54.386838 25108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:54.386868 25108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:54.388262 25108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:41873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:54.388587 25108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:54.388813 25108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:54.393790 25115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:54.393877 25117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:54.393965 25108 server_base.cc:1034] running on GCE node
W20260807 06:35:54.393798 25119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:54.394330 25108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:54.394661 25108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:54.395843 25108 hybrid_clock.cc:648] HybridClock initialized: now 1786084554395834 us; error 54 us; skew 500 ppm
Aug 07 06:35:54 dist-test-slave-rwrg krb5kdc[25086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084554, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:54.398629 25108 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:35:54.399076 25108 webserver.cc:460] Webserver started at http://127.2.243.62:36631/ using document root <none> and password file <none>
I20260807 06:35:54.399296 25108 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:54.399384 25108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:54.399578 25108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:54.400710 25108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "a7aef780888e45a7b9408d6bf52f73f8"
format_stamp: "Formatted at 2026-08-07 06:35:54 on dist-test-slave-rwrg"
server_key: "95cda6ed2079d7148750faeade6eefbb"
server_key_iv: "a43b97cd982b5d266430e0df66ff9af5"
server_key_version: "encryptionkey@0"
I20260807 06:35:54.401099 25108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "a7aef780888e45a7b9408d6bf52f73f8"
format_stamp: "Formatted at 2026-08-07 06:35:54 on dist-test-slave-rwrg"
server_key: "95cda6ed2079d7148750faeade6eefbb"
server_key_iv: "a43b97cd982b5d266430e0df66ff9af5"
server_key_version: "encryptionkey@0"
I20260807 06:35:54.402573 25108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:35:54.403409 25135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:54.403625 25108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:54.403738 25108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "a7aef780888e45a7b9408d6bf52f73f8"
format_stamp: "Formatted at 2026-08-07 06:35:54 on dist-test-slave-rwrg"
server_key: "95cda6ed2079d7148750faeade6eefbb"
server_key_iv: "a43b97cd982b5d266430e0df66ff9af5"
server_key_version: "encryptionkey@0"
I20260807 06:35:54.403810 25108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:54.417529 25108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:54.419997 25108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:54.420161 25108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:54.424047 25108 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:41873
I20260807 06:35:54.424084 25226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:41873 every 8 connection(s)
I20260807 06:35:54.424422 25108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:35:54.425297 25227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:54.427608 25227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8: Bootstrap starting.
I20260807 06:35:54.428213 25227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:54.428535 25227 log.cc:824] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:54.428746 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 25108
I20260807 06:35:54.428854 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:35:54.429026 3020 external_mini_cluster.cc:1428] Setting key bfe78cc70a53fd3ead7ad0c0f444c591
I20260807 06:35:54.429409 25227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8: No bootstrap required, opened a new log
I20260807 06:35:54.431481 25227 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7aef780888e45a7b9408d6bf52f73f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41873 } }
I20260807 06:35:54.431676 25227 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:54.431722 25227 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7aef780888e45a7b9408d6bf52f73f8, State: Initialized, Role: FOLLOWER
I20260807 06:35:54.431859 25227 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7aef780888e45a7b9408d6bf52f73f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41873 } }
I20260807 06:35:54.431950 25227 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:35:54.431988 25227 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:35:54.432037 25227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:54.432732 25227 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7aef780888e45a7b9408d6bf52f73f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41873 } }
I20260807 06:35:54.432834 25227 leader_election.cc:302] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7aef780888e45a7b9408d6bf52f73f8; no voters:
I20260807 06:35:54.432971 25227 leader_election.cc:288] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:35:54.433032 25232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:54.433189 25227 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:54.433231 25232 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [term 1 LEADER]: Becoming Leader. State: Replica: a7aef780888e45a7b9408d6bf52f73f8, State: Running, Role: LEADER
I20260807 06:35:54.433447 25232 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7aef780888e45a7b9408d6bf52f73f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41873 } }
I20260807 06:35:54.433843 25233 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7aef780888e45a7b9408d6bf52f73f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7aef780888e45a7b9408d6bf52f73f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41873 } } }
I20260807 06:35:54.433919 25233 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:54.434087 25234 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7aef780888e45a7b9408d6bf52f73f8. Latest consensus state: current_term: 1 leader_uuid: "a7aef780888e45a7b9408d6bf52f73f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7aef780888e45a7b9408d6bf52f73f8" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41873 } } }
I20260807 06:35:54.434140 25234 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:54.434538 25246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:35:54.435246 25246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:35:54.437007 25246 catalog_manager.cc:1347] Generated new cluster ID: 90c85a70312f435c999dd240ed0b7198
I20260807 06:35:54.437062 25246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:35:54 dist-test-slave-rwrg krb5kdc[25086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:54.441332 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:54.429536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.2.243.62:41873)
0807 06:35:54.429727 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:54.429737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:54.429774 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:35:54.430730 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:54.430739 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:54.430800 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:54.430946 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:54.431568 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:54.433327 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:54.434503 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:54.435112 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:54.435221 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:54.438034 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:54.438038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:54.438038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:54.438050 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:54.439799 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:54.440181 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:54.440184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:54.440184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:54.440273 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:54.440528 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:54.440528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:54.440529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:54.440765 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:54.440939 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:54.441054 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 06:35:54.441182 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260807 06:35:54.442293 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:35:54.451514 25246 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:35:54.451994 25246 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:35:54.456519 25246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7aef780888e45a7b9408d6bf52f73f8: Generated new TSK 0
I20260807 06:35:54.456691 25246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:35:54.496636 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 25108
2026-08-07T06:35:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1573 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:35:54 dist-test-slave-rwrg krb5kdc[25265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:35:54 dist-test-slave-rwrg krb5kdc[25265](info): set up 2 sockets
Aug 07 06:35:54 dist-test-slave-rwrg krb5kdc[25265](info): commencing operation
krb5kdc: starting...
W20260807 06:35:55.755453 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:35:55 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:35:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:35:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:35:55.879889 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45145
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:45145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:55.964468 25292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:55.964795 25292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:55.964833 25292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:55.966001 25292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:55.966066 25292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:55.966089 25292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:55.966110 25292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:55.966140 25292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:55.967619 25292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:45145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:55.967897 25292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:55.968125 25292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:55.973265 25301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:55.973264 25303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:55.973276 25306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:55.973585 25292 server_base.cc:1034] running on GCE node
I20260807 06:35:55.973815 25292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:55.974067 25292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:55.975205 25292 hybrid_clock.cc:648] HybridClock initialized: now 1786084555975194 us; error 31 us; skew 500 ppm
Aug 07 06:35:55 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084555, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:55.977986 25292 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:35:55.978404 25292 webserver.cc:460] Webserver started at http://127.2.243.62:44939/ using document root <none> and password file <none>
I20260807 06:35:55.978631 25292 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:55.978677 25292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:55.978786 25292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:55.979861 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "87a5b9be1263440e89df2a5861ad0225"
format_stamp: "Formatted at 2026-08-07 06:35:55 on dist-test-slave-rwrg"
server_key: "b392ea61417d268beac803f8e5606768"
server_key_iv: "f600c28bd02de1455b9a6768797f2059"
server_key_version: "encryptionkey@0"
I20260807 06:35:55.980222 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "87a5b9be1263440e89df2a5861ad0225"
format_stamp: "Formatted at 2026-08-07 06:35:55 on dist-test-slave-rwrg"
server_key: "b392ea61417d268beac803f8e5606768"
server_key_iv: "f600c28bd02de1455b9a6768797f2059"
server_key_version: "encryptionkey@0"
I20260807 06:35:55.981748 25292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:35:55.982595 25315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:55.982839 25292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:55.982954 25292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "87a5b9be1263440e89df2a5861ad0225"
format_stamp: "Formatted at 2026-08-07 06:35:55 on dist-test-slave-rwrg"
server_key: "b392ea61417d268beac803f8e5606768"
server_key_iv: "f600c28bd02de1455b9a6768797f2059"
server_key_version: "encryptionkey@0"
I20260807 06:35:55.983038 25292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:55.999017 25292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:56.001080 25292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:56.001255 25292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:56.005528 25292 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:45145
I20260807 06:35:56.005573 25394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:45145 every 8 connection(s)
I20260807 06:35:56.005944 25292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:35:56.006955 25396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:56.009392 25396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Bootstrap starting.
I20260807 06:35:56.009984 25396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:56.010279 25396 log.cc:824] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:56.011013 25396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: No bootstrap required, opened a new log
I20260807 06:35:56.012701 25396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:56.012835 25396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:56.012858 25396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a5b9be1263440e89df2a5861ad0225, State: Initialized, Role: FOLLOWER
I20260807 06:35:56.013012 25396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:56.013082 25396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:35:56.013113 25396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:35:56.013176 25396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:56.014025 25396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:56.014137 25396 leader_election.cc:302] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87a5b9be1263440e89df2a5861ad0225; no voters:
I20260807 06:35:56.014322 25396 leader_election.cc:288] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:35:56.014371 25399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:56.014557 25399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 LEADER]: Becoming Leader. State: Replica: 87a5b9be1263440e89df2a5861ad0225, State: Running, Role: LEADER
I20260807 06:35:56.014595 25396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:56.014712 25399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:56.015144 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 25292
I20260807 06:35:56.015136 25400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87a5b9be1263440e89df2a5861ad0225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } } }
I20260807 06:35:56.015149 25405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87a5b9be1263440e89df2a5861ad0225. Latest consensus state: current_term: 1 leader_uuid: "87a5b9be1263440e89df2a5861ad0225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } } }
I20260807 06:35:56.015239 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:35:56.015239 25400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:56.015249 25405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:56.015448 3020 external_mini_cluster.cc:1428] Setting key 99b8c04b6b570ca1c0e229d2cf4a4d42
I20260807 06:35:56.015465 25414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:35:56.016012 25414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:35:56.017804 25414 catalog_manager.cc:1347] Generated new cluster ID: 5207e1fde2984fcd98dd980231cb74d4
I20260807 06:35:56.017861 25414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:56.028117 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.015977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.2.243.62:45145)
0807 06:35:56.016135 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.016144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.016270 (+ 126us) server_negotiation.cc:407] Connection header received
0807 06:35:56.016418 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.016425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.016517 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.016670 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:56.017432 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.018923 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.020011 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.020644 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.020742 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.023743 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:56.023747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:56.023748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:56.023761 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:56.026396 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.026822 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.026825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.026826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.026957 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.027176 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.027177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.027178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.027490 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:56.027652 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.027731 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.027875 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260807 06:35:56.034966 25414 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:35:56.035699 25414 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:35:56.045771 25414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Generated new TSK 0
I20260807 06:35:56.046012 25414 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:35:56.075775 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45145
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:56.165232 25439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:56.165458 25439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:56.165481 25439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:56.166639 25439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:56.166684 25439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:56.166710 25439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:35:56.168207 25439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:56.168458 25439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:56.168746 25439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:56.174095 25450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:56.174135 25446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:56.174121 25452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:56.174285 25439 server_base.cc:1034] running on GCE node
I20260807 06:35:56.174479 25439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:56.174690 25439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:56.175837 25439 hybrid_clock.cc:648] HybridClock initialized: now 1786084556175824 us; error 31 us; skew 500 ppm
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:56.178618 25439 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:35:56.179064 25439 webserver.cc:460] Webserver started at http://127.2.243.1:36223/ using document root <none> and password file <none>
I20260807 06:35:56.179315 25439 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:56.179364 25439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:56.179492 25439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:56.180670 25439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "6b4f962797e14c919aa14f93a95b74f9"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "48d28b491911ef57a24495a8316f504e"
server_key_iv: "9d9fa8bbc9956b80ea00c48202f9b77e"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.181030 25439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "6b4f962797e14c919aa14f93a95b74f9"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "48d28b491911ef57a24495a8316f504e"
server_key_iv: "9d9fa8bbc9956b80ea00c48202f9b77e"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.182386 25439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:56.183182 25460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.183375 25439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:35:56.183481 25439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "6b4f962797e14c919aa14f93a95b74f9"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "48d28b491911ef57a24495a8316f504e"
server_key_iv: "9d9fa8bbc9956b80ea00c48202f9b77e"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.183555 25439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:56.198491 25439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:56.200837 25439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:56.201018 25439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:56.201251 25439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:56.201572 25439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:56.201629 25439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.201663 25439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:56.201689 25439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.207412 25439 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:33335
I20260807 06:35:56.207465 25639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:33335 every 8 connection(s)
I20260807 06:35:56.207777 25439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:35:56.211275 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 25439
I20260807 06:35:56.211413 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:35:56.211647 3020 external_mini_cluster.cc:1428] Setting key 62f8a163333bc57d886ebf821b457a64
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:56.217826 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.208547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:35713 (local address 127.2.243.62:45145)
0807 06:35:56.208619 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.208621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.208816 (+ 195us) server_negotiation.cc:407] Connection header received
0807 06:35:56.209327 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.209328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.209340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.209399 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:56.210297 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.210833 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.212078 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.212333 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.212412 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.214813 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:56.214820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:56.214820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:56.214828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:56.216289 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.216797 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.216798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.216798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.216858 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.217141 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.217142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.217142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.217260 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:56.217341 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.217636 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.217738 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:35:56.217983 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.208503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:35713)
0807 06:35:56.208682 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.208719 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.209187 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.209451 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.209454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.210013 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:56.210201 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.210206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.210915 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.210916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.211960 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.211964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.212418 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.212419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.212565 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.213135 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:56.213144 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:56.214709 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:56.216373 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:56.216380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:56.216383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:56.216719 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:56.216943 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:56.216944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:56.216944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:56.217063 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:56.217345 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:56.217352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:56.217490 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.217597 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.217756 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260807 06:35:56.218536 25640 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45145
I20260807 06:35:56.218645 25640 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:56.218833 25640 heartbeater.cc:511] Master 127.2.243.62:45145 requested a full tablet report, sending...
I20260807 06:35:56.219442 25341 ts_manager.cc:194] Registered new tserver with Master: 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1:33335)
I20260807 06:35:56.220378 25341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:35713
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:35:56.238754 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45145
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:56.320377 25648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:56.320622 25648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:56.320645 25648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:56.321877 25648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:56.321924 25648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:56.321951 25648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:35:56.323500 25648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:56.323740 25648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:56.323982 25648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:56.329160 25656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:56.329147 25657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:56.329195 25659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:56.329443 25648 server_base.cc:1034] running on GCE node
I20260807 06:35:56.329660 25648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:56.329882 25648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:56.331058 25648 hybrid_clock.cc:648] HybridClock initialized: now 1786084556331041 us; error 31 us; skew 500 ppm
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:56.333686 25648 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:35:56.334092 25648 webserver.cc:460] Webserver started at http://127.2.243.2:46297/ using document root <none> and password file <none>
I20260807 06:35:56.334276 25648 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:56.334316 25648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:56.334405 25648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:56.335415 25648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "a626be8a41e547e288d58326a062e763"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "1682a608b9d787056e9889edaa5b250c"
server_key_iv: "f63e5f2fc658bd9b4cef660a20b7986c"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.335733 25648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "a626be8a41e547e288d58326a062e763"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "1682a608b9d787056e9889edaa5b250c"
server_key_iv: "f63e5f2fc658bd9b4cef660a20b7986c"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.337144 25648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 06:35:56.337862 25670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.338073 25648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:56.338151 25648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "a626be8a41e547e288d58326a062e763"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "1682a608b9d787056e9889edaa5b250c"
server_key_iv: "f63e5f2fc658bd9b4cef660a20b7986c"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.338204 25648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:56.347883 25648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:56.349993 25648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:56.350119 25648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:56.350307 25648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:56.350553 25648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:56.350582 25648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.350605 25648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:56.350616 25648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.356810 25648 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:43205
I20260807 06:35:56.356859 25847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:43205 every 8 connection(s)
I20260807 06:35:56.357174 25648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:56.363962 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 25648
I20260807 06:35:56.364084 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:35:56.364341 3020 external_mini_cluster.cc:1428] Setting key 3ca88c2293fdad2f44b2a3c780710f26
I20260807 06:35:56.367147 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.357925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:53475 (local address 127.2.243.62:45145)
0807 06:35:56.357987 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.357989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.358175 (+ 186us) server_negotiation.cc:407] Connection header received
0807 06:35:56.358632 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.358633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.358642 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.358702 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:56.359615 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.360156 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.361365 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.361600 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.361660 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.363849 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:56.363851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:56.363851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:56.363860 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:56.365632 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.366072 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.366074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.366074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.366141 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.366379 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.366380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.366381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.366539 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:56.366610 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.366952 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.367051 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:35:56.367349 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.357856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:53475)
0807 06:35:56.358017 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.358088 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.358550 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.358751 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.358754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.359322 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:56.359505 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.359509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.360277 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.360279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.361267 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.361270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.361693 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.361694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.361816 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.362299 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:56.362307 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:56.363737 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:56.365741 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:56.365748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:56.365749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:56.365989 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:56.366211 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:56.366211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:56.366212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:56.366312 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:56.366629 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:56.366635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:56.366754 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.366927 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.367114 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260807 06:35:56.367823 25851 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45145
I20260807 06:35:56.367938 25851 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:56.368144 25851 heartbeater.cc:511] Master 127.2.243.62:45145 requested a full tablet report, sending...
I20260807 06:35:56.368587 25341 ts_manager.cc:194] Registered new tserver with Master: a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
I20260807 06:35:56.368981 25341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:53475
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:35:56.391973 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45145
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:56.476063 25859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:56.476322 25859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:56.476346 25859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:56.477699 25859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:56.477744 25859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:56.477771 25859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:35:56.479206 25859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:56.479449 25859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:56.479687 25859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:56.484787 25865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:56.484805 25866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:56.484805 25870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:56.485479 25859 server_base.cc:1034] running on GCE node
I20260807 06:35:56.485728 25859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:56.485988 25859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:56.487160 25859 hybrid_clock.cc:648] HybridClock initialized: now 1786084556487131 us; error 38 us; skew 500 ppm
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:56.489857 25859 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:35:56.490288 25859 webserver.cc:460] Webserver started at http://127.2.243.3:34889/ using document root <none> and password file <none>
I20260807 06:35:56.490479 25859 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:56.490525 25859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:56.490621 25859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:35:56.491712 25859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "81dedc1957b7412fbefc4c2de268fa30"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "30c583f0ab94f42fd218e21921269c58"
server_key_iv: "f20cf8d3592c7db02a8e2b773deebaed"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.492043 25859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "81dedc1957b7412fbefc4c2de268fa30"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "30c583f0ab94f42fd218e21921269c58"
server_key_iv: "f20cf8d3592c7db02a8e2b773deebaed"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.493559 25859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:35:56.494318 25878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.494489 25859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:35:56.494577 25859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "81dedc1957b7412fbefc4c2de268fa30"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "30c583f0ab94f42fd218e21921269c58"
server_key_iv: "f20cf8d3592c7db02a8e2b773deebaed"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.494632 25859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:56.501590 25859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:56.503609 25859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:56.503732 25859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:56.503933 25859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:56.504184 25859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:56.504216 25859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.504259 25859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:56.504297 25859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.510826 25859 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:41419
I20260807 06:35:56.510897 26061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:41419 every 8 connection(s)
I20260807 06:35:56.511250 25859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:35:56.517428 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 25859
I20260807 06:35:56.517537 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:35:56.517720 3020 external_mini_cluster.cc:1428] Setting key 1aefa9da81bede05f832c8330b0cb672
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:56.521333 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.512082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:39703 (local address 127.2.243.62:45145)
0807 06:35:56.512178 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.512180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.512355 (+ 175us) server_negotiation.cc:407] Connection header received
0807 06:35:56.512931 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.512932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.512941 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.512995 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:56.513931 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.514433 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.515719 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.515918 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.515980 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.518481 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:56.518482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:56.518482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:56.518488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:56.519880 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.520333 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.520334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.520334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.520412 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.520722 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.520723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.520723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.520834 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:56.520892 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.521202 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.521270 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:35:56.521495 26066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.512022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:39703)
0807 06:35:56.512227 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.512265 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.512835 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.513052 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.513056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.513647 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:56.513834 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.513839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.514534 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.514535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.515538 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.515541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.516000 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.516001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.516117 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.516721 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:56.516730 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:56.518367 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:56.519971 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:56.519977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:56.519979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:56.520234 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:56.520478 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:56.520479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:56.520479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:56.520658 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:56.520915 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:56.520922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:56.521077 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.521158 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.521278 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
I20260807 06:35:56.522025 26063 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45145
I20260807 06:35:56.522131 26063 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:56.522306 26063 heartbeater.cc:511] Master 127.2.243.62:45145 requested a full tablet report, sending...
I20260807 06:35:56.522713 25341 ts_manager.cc:194] Registered new tserver with Master: 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419)
I20260807 06:35:56.523178 25341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:39703
I20260807 06:35:56.529989 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:35:56.538012 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.531393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.2.243.62:45145)
0807 06:35:56.531466 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.531468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.531583 (+ 115us) server_negotiation.cc:407] Connection header received
0807 06:35:56.531685 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.531686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.531699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.531764 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:56.532387 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.532983 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.533791 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.534031 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.534122 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.535413 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:56.535415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:56.535415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:56.535423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:56.536780 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.537141 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.537142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.537142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.537205 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:56.537411 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:56.537412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:56.537412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:56.537572 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:56.537680 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.537820 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.537902 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:35:56.539515 25341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:35:56.541232 25341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:35:56.547576 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.543430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:33335 (local address 127.0.0.1:53224)
0807 06:35:56.543601 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.543640 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.543700 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.544034 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.544146 (+ 112us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.544149 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:56.544218 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.544220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.545276 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.545277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.546434 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.546437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.547255 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.547256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.547333 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.547346 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.547452 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.547510 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":56,"threads_started":1}
I20260807 06:35:56.547588 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.543547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.2.243.1:33335)
0807 06:35:56.543856 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.543860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.543870 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:35:56.543913 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.543914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.543927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.544027 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:56.544308 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.545164 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.546527 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.547171 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.547255 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.547317 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.547469 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.547507 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
I20260807 06:35:56.547914 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.543430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:43205 (local address 127.0.0.1:42220)
0807 06:35:56.543991 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.544001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.544063 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.544249 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.544250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.544252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:56.544345 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.544347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.545767 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.545769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.546877 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.546880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.547716 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.547718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.547776 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.547780 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.547807 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.547844 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":3,"thread_start_us":55,"threads_started":1}
I20260807 06:35:56.548207 26072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.543622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:41419 (local address 127.0.0.1:53844)
0807 06:35:56.544127 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.544135 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.544179 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.544410 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.544411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.544412 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:56.544447 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.544448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.545736 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.545737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.546722 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.546724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.547870 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.547871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.547924 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.547928 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.547984 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.548146 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":54,"threads_started":1}
I20260807 06:35:56.548220 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.543803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.2.243.3:41419)
0807 06:35:56.543980 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.543985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.544233 (+ 248us) server_negotiation.cc:407] Connection header received
0807 06:35:56.544272 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.544274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.544300 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.544401 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:56.544533 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.545622 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.546817 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.547500 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.547577 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.547630 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.548101 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.548144 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260807 06:35:56.548388 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.543668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.2.243.2:43205)
0807 06:35:56.543810 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.543815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.544064 (+ 249us) server_negotiation.cc:407] Connection header received
0807 06:35:56.544128 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.544130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.544150 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.544466 (+ 316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:56.544564 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.545644 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.546962 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.547612 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.548141 (+ 529us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.548210 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.548238 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.548286 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260807 06:35:56.548594 25532 tablet_service.cc:1467] Processing CreateTablet for tablet 53e7461b302e499e9ed634725ebf9952 (DEFAULT_TABLE table=test-table [id=bc092cfa4fb04b5cb6552a0ec6a3274e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:56.548977 25532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53e7461b302e499e9ed634725ebf9952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:56.549054 25745 tablet_service.cc:1467] Processing CreateTablet for tablet 53e7461b302e499e9ed634725ebf9952 (DEFAULT_TABLE table=test-table [id=bc092cfa4fb04b5cb6552a0ec6a3274e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:56.549400 25745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53e7461b302e499e9ed634725ebf9952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:56.551841 26082 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap starting.
I20260807 06:35:56.552413 25960 tablet_service.cc:1467] Processing CreateTablet for tablet 53e7461b302e499e9ed634725ebf9952 (DEFAULT_TABLE table=test-table [id=bc092cfa4fb04b5cb6552a0ec6a3274e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:56.552632 26082 tablet_bootstrap.cc:654] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:56.552750 26083 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Bootstrap starting.
I20260807 06:35:56.552785 25960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53e7461b302e499e9ed634725ebf9952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:56.553277 26082 log.cc:824] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:56.553315 26083 tablet_bootstrap.cc:654] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:56.553961 26083 log.cc:824] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:56.554164 26082 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: No bootstrap required, opened a new log
I20260807 06:35:56.554289 26082 ts_tablet_manager.cc:1397] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 06:35:56.554678 26083 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: No bootstrap required, opened a new log
I20260807 06:35:56.554826 26083 ts_tablet_manager.cc:1397] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:35:56.555980 26087 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Bootstrap starting.
I20260807 06:35:56.556417 26082 raft_consensus.cc:348] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.556653 26082 raft_consensus.cc:374] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:56.556720 26082 raft_consensus.cc:729] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f962797e14c919aa14f93a95b74f9, State: Initialized, Role: FOLLOWER
I20260807 06:35:56.556784 26087 tablet_bootstrap.cc:654] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:56.556671 26083 raft_consensus.cc:348] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.556842 26083 raft_consensus.cc:374] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:56.556891 26083 raft_consensus.cc:729] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a626be8a41e547e288d58326a062e763, State: Initialized, Role: FOLLOWER
I20260807 06:35:56.556847 26082 consensus_queue.cc:260] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.557030 26083 consensus_queue.cc:260] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.557140 26082 ts_tablet_manager.cc:1428] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:35:56.557152 25640 heartbeater.cc:503] Master 127.2.243.62:45145 was elected leader, sending a full tablet report...
I20260807 06:35:56.557493 26083 ts_tablet_manager.cc:1428] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:35:56.557495 26087 log.cc:824] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:56.557542 25851 heartbeater.cc:503] Master 127.2.243.62:45145 was elected leader, sending a full tablet report...
I20260807 06:35:56.558394 26087 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: No bootstrap required, opened a new log
I20260807 06:35:56.558465 26087 ts_tablet_manager.cc:1397] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:35:56.559891 26087 raft_consensus.cc:348] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.560015 26087 raft_consensus.cc:374] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:56.560038 26087 raft_consensus.cc:729] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81dedc1957b7412fbefc4c2de268fa30, State: Initialized, Role: FOLLOWER
I20260807 06:35:56.560139 26087 consensus_queue.cc:260] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.560381 26087 ts_tablet_manager.cc:1428] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:35:56.560525 26063 heartbeater.cc:503] Master 127.2.243.62:45145 was elected leader, sending a full tablet report...
W20260807 06:35:56.607707 25852 tablet.cc:2369] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:35:56.708540 25641 tablet.cc:2369] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:35:56.748891 26091 raft_consensus.cc:482] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:35:56.749028 26091 raft_consensus.cc:504] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.749482 26091 leader_election.cc:288] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a626be8a41e547e288d58326a062e763 (127.2.243.2:43205), 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1:33335)
I20260807 06:35:56.753733 26066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.749742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:33335 (local address 127.2.243.3:38369)
0807 06:35:56.749812 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.749828 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.749897 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.750062 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.750063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.750065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:56.750148 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.750151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.751224 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.751225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.752697 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.752700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.753446 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.753447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.753561 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.753567 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.753618 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.753643 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":75}
I20260807 06:35:56.753746 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.749851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:38369 (local address 127.2.243.1:33335)
0807 06:35:56.749910 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.749912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.749924 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:35:56.749960 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.749961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.749980 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.750036 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:56.750336 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.751124 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.752808 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.753357 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.753435 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.753446 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.753651 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.753688 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:35:56.753746 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.749847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:53501 (local address 127.2.243.2:43205)
0807 06:35:56.749891 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.749893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.750119 (+ 226us) server_negotiation.cc:407] Connection header received
0807 06:35:56.750147 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.750147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.750159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.750341 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:56.750464 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.751148 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.752808 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.753357 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.753435 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.753447 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.753636 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.753676 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:35:56.753741 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.749748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:43205 (local address 127.2.243.3:53501)
0807 06:35:56.750007 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.750033 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:56.750093 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:56.750342 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:56.750342 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:56.750343 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:56.750396 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.750397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.751224 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.751225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.752697 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:56.752700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.753446 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.753447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:56.753525 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.753529 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:35:56.753593 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:35:56.753638 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":236,"thread_start_us":104,"threads_started":1}
I20260807 06:35:56.754132 25778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e7461b302e499e9ed634725ebf9952" candidate_uuid: "81dedc1957b7412fbefc4c2de268fa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a626be8a41e547e288d58326a062e763" is_pre_election: true
I20260807 06:35:56.754127 25565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e7461b302e499e9ed634725ebf9952" candidate_uuid: "81dedc1957b7412fbefc4c2de268fa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4f962797e14c919aa14f93a95b74f9" is_pre_election: true
I20260807 06:35:56.754283 25565 raft_consensus.cc:2454] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81dedc1957b7412fbefc4c2de268fa30 in term 0.
I20260807 06:35:56.754283 25778 raft_consensus.cc:2454] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81dedc1957b7412fbefc4c2de268fa30 in term 0.
I20260807 06:35:56.754511 25882 leader_election.cc:302] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81dedc1957b7412fbefc4c2de268fa30, a626be8a41e547e288d58326a062e763; no voters:
I20260807 06:35:56.754689 26091 raft_consensus.cc:2781] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:35:56.754758 26091 raft_consensus.cc:482] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:35:56.754776 26091 raft_consensus.cc:3037] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:56.755517 26091 raft_consensus.cc:504] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.755661 26091 leader_election.cc:288] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [CANDIDATE]: Term 1 election: Requested vote from peers a626be8a41e547e288d58326a062e763 (127.2.243.2:43205), 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1:33335)
I20260807 06:35:56.755841 25778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e7461b302e499e9ed634725ebf9952" candidate_uuid: "81dedc1957b7412fbefc4c2de268fa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a626be8a41e547e288d58326a062e763"
I20260807 06:35:56.755846 25565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e7461b302e499e9ed634725ebf9952" candidate_uuid: "81dedc1957b7412fbefc4c2de268fa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4f962797e14c919aa14f93a95b74f9"
I20260807 06:35:56.755923 25565 raft_consensus.cc:3037] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:56.755923 25778 raft_consensus.cc:3037] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:56.756868 25565 raft_consensus.cc:2454] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81dedc1957b7412fbefc4c2de268fa30 in term 1.
I20260807 06:35:56.756868 25778 raft_consensus.cc:2454] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81dedc1957b7412fbefc4c2de268fa30 in term 1.
I20260807 06:35:56.757098 25882 leader_election.cc:302] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81dedc1957b7412fbefc4c2de268fa30, a626be8a41e547e288d58326a062e763; no voters:
I20260807 06:35:56.757261 26091 raft_consensus.cc:2781] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:56.757428 26091 raft_consensus.cc:686] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 LEADER]: Becoming Leader. State: Replica: 81dedc1957b7412fbefc4c2de268fa30, State: Running, Role: LEADER
I20260807 06:35:56.757524 26091 consensus_queue.cc:237] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:56.758376 25341 catalog_manager.cc:5495] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3). New cstate: current_term: 1 leader_uuid: "81dedc1957b7412fbefc4c2de268fa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } health_report { overall_health: HEALTHY } } }
W20260807 06:35:56.761847 26064 tablet.cc:2369] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:35:56.772814 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.770635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.2.243.3:41419)
0807 06:35:56.770680 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:56.770682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:56.770690 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:35:56.770743 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:56.770744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:56.770760 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:56.770814 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:35:56.770914 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.771367 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.772087 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:56.772265 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:56.772323 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:56.772456 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:35:56.772638 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:35:56.772711 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:56.772744 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 06:35:56.772762 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:35:56.774416 25565 raft_consensus.cc:1261] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Refusing update from remote peer 81dedc1957b7412fbefc4c2de268fa30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:56.774422 25778 raft_consensus.cc:1261] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Refusing update from remote peer 81dedc1957b7412fbefc4c2de268fa30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:56.774793 26093 consensus_queue.cc:1035] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:56.774930 26091 consensus_queue.cc:1035] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:56.777321 26097 mvcc.cc:205] Tried to move back new op lower bound from 7315802344545533952 to 7315802344479367168. Current Snapshot: MvccSnapshot[applied={T|T < 7315802344545533952}]
I20260807 06:35:56.778427 26102 mvcc.cc:205] Tried to move back new op lower bound from 7315802344545533952 to 7315802344479367168. Current Snapshot: MvccSnapshot[applied={T|T < 7315802344545533952}]
I20260807 06:35:56.778409 26100 mvcc.cc:205] Tried to move back new op lower bound from 7315802344545533952 to 7315802344479367168. Current Snapshot: MvccSnapshot[applied={T|T < 7315802344545533952 or (T in {7315802344545533952})}]
I20260807 06:35:56.779819 25341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:56.779930 25341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:56.780797 25341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Sending DeleteTablet for 3 replicas of tablet 53e7461b302e499e9ed634725ebf9952
I20260807 06:35:56.781191 25745 tablet_service.cc:1514] Processing DeleteTablet for tablet 53e7461b302e499e9ed634725ebf9952 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:56 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:42220
I20260807 06:35:56.781236 25532 tablet_service.cc:1514] Processing DeleteTablet for tablet 53e7461b302e499e9ed634725ebf9952 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:56 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:53224
I20260807 06:35:56.781260 25960 tablet_service.cc:1514] Processing DeleteTablet for tablet 53e7461b302e499e9ed634725ebf9952 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:56 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:53844
I20260807 06:35:56.781448 26116 tablet_replica.cc:330] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: stopping tablet replica
I20260807 06:35:56.781477 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 25439
I20260807 06:35:56.781566 26116 raft_consensus.cc:2229] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:56.781726 26116 raft_consensus.cc:2258] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:56.786005 26117 tablet_replica.cc:330] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: stopping tablet replica
I20260807 06:35:56.786201 26117 raft_consensus.cc:2229] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:35:56.786331 26117 raft_consensus.cc:2258] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:56.788241 26117 ts_tablet_manager.cc:1905] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:35:56.792351 26117 ts_tablet_manager.cc:1918] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:56.792451 26117 log.cc:1196] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/53e7461b302e499e9ed634725ebf9952
I20260807 06:35:56.792780 26117 ts_tablet_manager.cc:1939] T 53e7461b302e499e9ed634725ebf9952 P 81dedc1957b7412fbefc4c2de268fa30: Deleting consensus metadata
I20260807 06:35:56.796653 25316 catalog_manager.cc:4841] TS 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419): tablet 53e7461b302e499e9ed634725ebf9952 (table test-table [id=bc092cfa4fb04b5cb6552a0ec6a3274e]) successfully deleted
I20260807 06:35:56.797031 26115 tablet_replica.cc:330] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: stopping tablet replica
I20260807 06:35:56.797179 26115 raft_consensus.cc:2229] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:56.797317 26115 raft_consensus.cc:2258] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:56.797763 26115 ts_tablet_manager.cc:1905] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:35:56.799261 26115 ts_tablet_manager.cc:1918] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:56.799345 26115 log.cc:1196] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/53e7461b302e499e9ed634725ebf9952
I20260807 06:35:56.799614 26115 ts_tablet_manager.cc:1939] T 53e7461b302e499e9ed634725ebf9952 P a626be8a41e547e288d58326a062e763: Deleting consensus metadata
I20260807 06:35:56.800089 25318 catalog_manager.cc:4841] TS a626be8a41e547e288d58326a062e763 (127.2.243.2:43205): tablet 53e7461b302e499e9ed634725ebf9952 (table test-table [id=bc092cfa4fb04b5cb6552a0ec6a3274e]) successfully deleted
W20260807 06:35:56.801540 25316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:33335 (error 108)
I20260807 06:35:56.802039 26072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:56.801914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:33335 (local address 127.0.0.1:53234)
0807 06:35:56.801969 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:56.802012 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:33335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 06:35:56.802130 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 25648
W20260807 06:35:56.802259 25316 catalog_manager.cc:4584] TS 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1:33335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53e7461b302e499e9ed634725ebf9952: Network error: Client connection negotiation failed: client connection to 127.2.243.1:33335: connect: Connection refused (error 111)
I20260807 06:35:56.831578 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 25859
I20260807 06:35:56.840811 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 25292
I20260807 06:35:56.850708 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45145
--webserver_interface=127.2.243.62
--webserver_port=44939
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:45145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:56.934453 26120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:56.934662 26120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:56.934684 26120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:56.935814 26120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:56.935856 26120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:56.935870 26120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:56.935883 26120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:56.935935 26120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:56.937505 26120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:45145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:56.937760 26120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:56.938005 26120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:56.947067 26134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:56.947062 26137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:56.947350 26120 server_base.cc:1034] running on GCE node
W20260807 06:35:56.947062 26132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:56.947680 26120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:56.947953 26120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:56.949095 26120 hybrid_clock.cc:648] HybridClock initialized: now 1786084556949075 us; error 33 us; skew 500 ppm
Aug 07 06:35:56 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:56.957973 26120 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:35:56.958503 26120 webserver.cc:460] Webserver started at http://127.2.243.62:44939/ using document root <none> and password file <none>
I20260807 06:35:56.958797 26120 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:56.958858 26120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:56.960321 26120 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:35:56.961117 26150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.961297 26120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:56.961413 26120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "87a5b9be1263440e89df2a5861ad0225"
format_stamp: "Formatted at 2026-08-07 06:35:55 on dist-test-slave-rwrg"
server_key: "b392ea61417d268beac803f8e5606768"
server_key_iv: "f600c28bd02de1455b9a6768797f2059"
server_key_version: "encryptionkey@0"
I20260807 06:35:56.961838 26120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:56.979140 26120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:56.981134 26120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:56.981307 26120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:56.985497 26120 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:45145
I20260807 06:35:56.985555 26229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:45145 every 8 connection(s)
I20260807 06:35:56.985941 26120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:35:56.987547 26231 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:35:56.988143 26231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Bootstrap starting.
I20260807 06:35:56.996268 26231 log.cc:824] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:56.996277 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 26120
I20260807 06:35:56.996658 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:33335
--local_ip_for_outbound_sockets=127.2.243.1
--tserver_master_addrs=127.2.243.62:45145
--webserver_port=36223
--webserver_interface=127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:35:56.998317 26231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:35:56.998634 26231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Bootstrap complete.
I20260807 06:35:57.001081 26231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.001331 26231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a5b9be1263440e89df2a5861ad0225, State: Initialized, Role: FOLLOWER
I20260807 06:35:57.001538 26231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.001636 26231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:35:57.001694 26231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:35:57.001744 26231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:35:57.002723 26231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.002842 26231 leader_election.cc:302] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87a5b9be1263440e89df2a5861ad0225; no voters:
I20260807 06:35:57.003048 26231 leader_election.cc:288] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:35:57.003083 26235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:35:57.003192 26235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 LEADER]: Becoming Leader. State: Replica: 87a5b9be1263440e89df2a5861ad0225, State: Running, Role: LEADER
I20260807 06:35:57.003321 26231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:57.003353 26235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.003712 26236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "87a5b9be1263440e89df2a5861ad0225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } } }
I20260807 06:35:57.003723 26238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87a5b9be1263440e89df2a5861ad0225. Latest consensus state: current_term: 2 leader_uuid: "87a5b9be1263440e89df2a5861ad0225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } } }
I20260807 06:35:57.003820 26236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:57.003834 26238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:57.004031 26245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:35:57.004947 26245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:35:57.005110 26245 catalog_manager.cc:1259] Loaded cluster ID: 5207e1fde2984fcd98dd980231cb74d4
I20260807 06:35:57.005375 26245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:35:57.005741 26245 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 06:35:57.010959 26245 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 06:35:57.013448 26245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:35:57.013657 26245 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Loaded TSK: 0
I20260807 06:35:57.013877 26245 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:35:57.081450 26234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:57.081693 26234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:57.081733 26234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:57.082911 26234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:57.082979 26234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:57.083024 26234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:35:57.084470 26234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:33335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=36223
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:57.084753 26234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:57.084980 26234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:57.090916 26277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.090932 26273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.091150 26234 server_base.cc:1034] running on GCE node
W20260807 06:35:57.091003 26272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.091444 26234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:57.091715 26234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:57.092888 26234 hybrid_clock.cc:648] HybridClock initialized: now 1786084557092857 us; error 43 us; skew 500 ppm
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:57.095861 26234 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:35:57.096349 26234 webserver.cc:460] Webserver started at http://127.2.243.1:36223/ using document root <none> and password file <none>
I20260807 06:35:57.096619 26234 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:57.096705 26234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:57.098133 26234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.098896 26287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.099084 26234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.099200 26234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "6b4f962797e14c919aa14f93a95b74f9"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "48d28b491911ef57a24495a8316f504e"
server_key_iv: "9d9fa8bbc9956b80ea00c48202f9b77e"
server_key_version: "encryptionkey@0"
I20260807 06:35:57.099531 26234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:57.126811 26234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:57.129284 26234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:57.129493 26234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:57.129760 26234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:57.130352 26295 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:35:57.131975 26234 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:35:57.132028 26234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:35:57.132078 26234 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:35:57.132829 26234 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:35:57.132874 26234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:35:57.132916 26295 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap starting.
I20260807 06:35:57.140807 26234 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:33335
I20260807 06:35:57.140847 26467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:33335 every 8 connection(s)
I20260807 06:35:57.141352 26234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:35:57.142833 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 26234
I20260807 06:35:57.143143 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:43205
--local_ip_for_outbound_sockets=127.2.243.2
--tserver_master_addrs=127.2.243.62:45145
--webserver_port=46297
--webserver_interface=127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:35:57.144366 26295 log.cc:824] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:57.146407 26295 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:35:57.146701 26295 tablet_bootstrap.cc:492] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap complete.
I20260807 06:35:57.147907 26295 ts_tablet_manager.cc:1397] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:57.149297 26295 raft_consensus.cc:348] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:57.149650 26295 raft_consensus.cc:729] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f962797e14c919aa14f93a95b74f9, State: Initialized, Role: FOLLOWER
I20260807 06:35:57.149859 26295 consensus_queue.cc:260] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } } peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } }
I20260807 06:35:57.150169 26295 ts_tablet_manager.cc:1428] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Time spent starting tablet: real 0.002s user 0.005s sys 0.001s
I20260807 06:35:57.154356 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.142205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:38063)
0807 06:35:57.142403 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.142457 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.143285 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.143867 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.143872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.144314 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:57.144626 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.144633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.145889 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.145889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.146801 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.146804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.147674 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.147675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.147790 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.148426 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:57.148434 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:57.150169 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:57.152741 (+ 2572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.152747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.152749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.152972 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.153296 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.153297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.153297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.153394 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.153901 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:57.153906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:57.154013 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.154093 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.154216 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260807 06:35:57.154621 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.142333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:38063 (local address 127.2.243.62:45145)
0807 06:35:57.142564 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.142574 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.142628 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:35:57.143549 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.143560 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.143629 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.143810 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:57.144801 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.145758 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.146899 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.147574 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.147686 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.150306 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:57.150311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:57.150311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:57.150326 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:57.152643 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.153067 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.153071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.153071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.153211 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.153466 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.153467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.153468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.153819 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:57.153930 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.154194 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.154360 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260807 06:35:57.155297 26469 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45145
I20260807 06:35:57.155414 26469 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:57.155652 26469 heartbeater.cc:511] Master 127.2.243.62:45145 requested a full tablet report, sending...
I20260807 06:35:57.156481 26179 ts_manager.cc:194] Registered new tserver with Master: 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1:33335)
I20260807 06:35:57.157701 26179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:38063
I20260807 06:35:57.158533 26469 heartbeater.cc:503] Master 127.2.243.62:45145 was elected leader, sending a full tablet report...
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084556, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:35:57.165845 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.157122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:33335 (local address 127.0.0.1:53246)
0807 06:35:57.157335 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.157364 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.157420 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.157631 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.157634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.157952 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:57.158028 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.158031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.159389 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.159389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.159929 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.159931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.160630 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.160631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.160685 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.161174 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:57.161178 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:57.162700 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:57.164720 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.164726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.164728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.164913 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.165145 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.165146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.165146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.165224 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.165537 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:57.165542 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:57.165584 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.165684 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.165777 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":102,"threads_started":1}
I20260807 06:35:57.165951 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.157227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53246 (local address 127.2.243.1:33335)
0807 06:35:57.157413 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.157418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.157432 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:35:57.157508 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.157511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.157528 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.157620 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:57.158109 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.159299 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.160018 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.160542 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.160635 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.162795 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:57.162803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:57.162804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:57.162812 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:57.164639 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.164998 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.165001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.165002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.165071 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.165272 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.165272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.165273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.165452 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:57.165562 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.165739 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.165845 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260807 06:35:57.166255 26361 tablet_service.cc:1514] Processing DeleteTablet for tablet 53e7461b302e499e9ed634725ebf9952 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:56 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:53246
I20260807 06:35:57.166877 26483 tablet_replica.cc:330] stopping tablet replica
I20260807 06:35:57.167001 26483 raft_consensus.cc:2229] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:57.167083 26483 raft_consensus.cc:2258] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:57.167405 26483 ts_tablet_manager.cc:1905] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:35:57.169502 26483 ts_tablet_manager.cc:1918] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:57.169595 26483 log.cc:1196] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/53e7461b302e499e9ed634725ebf9952
I20260807 06:35:57.169821 26483 ts_tablet_manager.cc:1939] T 53e7461b302e499e9ed634725ebf9952 P 6b4f962797e14c919aa14f93a95b74f9: Deleting consensus metadata
I20260807 06:35:57.170312 26151 catalog_manager.cc:4841] TS 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1:33335): tablet 53e7461b302e499e9ed634725ebf9952 (table test-table [id=bc092cfa4fb04b5cb6552a0ec6a3274e]) successfully deleted
W20260807 06:35:57.237483 26474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:57.237704 26474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:57.237727 26474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:57.238938 26474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:57.238981 26474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:57.239008 26474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:35:57.240463 26474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:43205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=46297
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:57.240732 26474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:57.240933 26474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:57.246085 26493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.246001 26498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.246014 26474 server_base.cc:1034] running on GCE node
W20260807 06:35:57.245998 26496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.246526 26474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:57.246760 26474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:57.247926 26474 hybrid_clock.cc:648] HybridClock initialized: now 1786084557247906 us; error 36 us; skew 500 ppm
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:57.250579 26474 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:35:57.251030 26474 webserver.cc:460] Webserver started at http://127.2.243.2:46297/ using document root <none> and password file <none>
I20260807 06:35:57.251210 26474 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:57.251266 26474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:57.252677 26474 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.253216 26507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.253417 26474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.253502 26474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "a626be8a41e547e288d58326a062e763"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "1682a608b9d787056e9889edaa5b250c"
server_key_iv: "f63e5f2fc658bd9b4cef660a20b7986c"
server_key_version: "encryptionkey@0"
I20260807 06:35:57.253800 26474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:57.264400 26474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:57.266762 26474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:57.266865 26474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:57.267028 26474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:57.267271 26474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:57.267302 26474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.267321 26474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:57.267333 26474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.273192 26474 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:43205
I20260807 06:35:57.273231 26672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:43205 every 8 connection(s)
I20260807 06:35:57.273566 26474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:35:57.273741 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 26474
I20260807 06:35:57.274052 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:41419
--local_ip_for_outbound_sockets=127.2.243.3
--tserver_master_addrs=127.2.243.62:45145
--webserver_port=34889
--webserver_interface=127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:57.285282 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.274383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:57979 (local address 127.2.243.62:45145)
0807 06:35:57.274460 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.274463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.275817 (+ 1354us) server_negotiation.cc:407] Connection header received
0807 06:35:57.276349 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.276351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.276364 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.276440 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:57.277416 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.278224 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.279396 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.279781 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.279863 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.282008 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:57.282019 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:57.282019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:57.282028 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:57.283740 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.284131 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.284133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.284134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.284205 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.284449 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.284450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.284450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.284658 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:57.284743 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.285036 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.285148 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:35:57.285292 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.274372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:57979)
0807 06:35:57.275670 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.275708 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.276236 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.276535 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.276539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.277123 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:57.277308 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.277312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.278330 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.278332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.279300 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.279303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.279880 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.279880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.279979 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.280552 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:57.280561 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:57.281910 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:57.283836 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.283843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.283844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.284057 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.284281 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.284282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.284282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.284379 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.284759 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:57.284766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:57.284887 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.284987 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.285126 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"thread_start_us":86,"threads_started":1}
I20260807 06:35:57.285915 26674 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45145
I20260807 06:35:57.286028 26674 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:57.286257 26674 heartbeater.cc:511] Master 127.2.243.62:45145 requested a full tablet report, sending...
I20260807 06:35:57.286686 26179 ts_manager.cc:194] Registered new tserver with Master: a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
I20260807 06:35:57.287070 26179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:57979
W20260807 06:35:57.358614 26679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:57.358852 26679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:57.358891 26679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:57.360049 26679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:57.360113 26679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:57.360155 26679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:35:57.361557 26679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:41419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=34889
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:57.361838 26679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:57.362082 26679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:57.367277 26688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.367278 26693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.367298 26689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.367388 26679 server_base.cc:1034] running on GCE node
I20260807 06:35:57.367810 26679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:57.368057 26679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:57.369215 26679 hybrid_clock.cc:648] HybridClock initialized: now 1786084557369196 us; error 31 us; skew 500 ppm
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:57.371884 26679 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:35:57.372401 26679 webserver.cc:460] Webserver started at http://127.2.243.3:34889/ using document root <none> and password file <none>
I20260807 06:35:57.372663 26679 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:57.372762 26679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:57.374154 26679 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:57.374811 26704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.375030 26679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:57.375151 26679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "81dedc1957b7412fbefc4c2de268fa30"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "30c583f0ab94f42fd218e21921269c58"
server_key_iv: "f20cf8d3592c7db02a8e2b773deebaed"
server_key_version: "encryptionkey@0"
I20260807 06:35:57.375506 26679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:57.396780 26679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:57.398766 26679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:57.398931 26679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:57.399176 26679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:57.399489 26679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:57.399542 26679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.399578 26679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:57.399626 26679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.406390 26679 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:41419
I20260807 06:35:57.406435 26863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:41419 every 8 connection(s)
I20260807 06:35:57.406770 26679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:35:57.410863 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 26679
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:35:57.417059 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.407464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:42229 (local address 127.2.243.62:45145)
0807 06:35:57.407524 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.407527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.407742 (+ 215us) server_negotiation.cc:407] Connection header received
0807 06:35:57.408286 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.408288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.408301 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.408351 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:57.409358 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.409835 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.410870 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.411160 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.411225 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.413586 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:57.413587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:57.413588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:57.413597 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:57.415416 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.415960 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.415962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.415962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.416042 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.416269 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.416270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.416271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.416376 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:57.416464 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.416795 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.416960 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 06:35:57.417143 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.407445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:42229)
0807 06:35:57.407625 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.407660 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.408173 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.408416 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.408420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.409054 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:57.409251 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.409256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.409927 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.409928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.410766 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.410770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.411274 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.411275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.411430 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.412019 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:57.412033 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:57.413495 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:57.415505 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.415511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.415513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.415881 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.416105 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:57.416106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:57.416106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:57.416207 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:57.416469 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:57.416475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:57.416656 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.416758 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.416923 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260807 06:35:57.417608 26864 heartbeater.cc:347] Connected to a master server at 127.2.243.62:45145
I20260807 06:35:57.417724 26864 heartbeater.cc:464] Registering TS with master...
I20260807 06:35:57.417924 26864 heartbeater.cc:511] Master 127.2.243.62:45145 requested a full tablet report, sending...
I20260807 06:35:57.418303 26179 ts_manager.cc:194] Registered new tserver with Master: 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419)
I20260807 06:35:57.418465 26870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.411309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.2.243.62:45145)
0807 06:35:57.411425 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.411427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.411536 (+ 109us) server_negotiation.cc:407] Connection header received
0807 06:35:57.411616 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.411617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.411626 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.411709 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:57.412322 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.412862 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.413775 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.414071 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.414135 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.415571 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:57.415572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:57.415573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:57.415580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:57.417084 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.417582 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.417582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.417583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.417634 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:57.417860 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:57.417861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:57.417862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:57.417975 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:57.418151 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.418266 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.418352 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260807 06:35:57.418815 26179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:42229
I20260807 06:35:57.419308 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:35:57.421834 26870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.419715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.2.243.62:45145)
0807 06:35:57.419769 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.419771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.419875 (+ 104us) server_negotiation.cc:407] Connection header received
0807 06:35:57.419931 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.419932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.419957 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.420017 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:35:57.420094 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.420554 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.421251 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.421421 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.421469 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.421611 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:35:57.421692 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:35:57.421745 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.421776 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.421791 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:35:57.422082 26179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:35:57.423594 26179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:35:57.425928 26361 tablet_service.cc:1467] Processing CreateTablet for tablet d1b4d51234ba407dbe191c24bbe6c0a7 (DEFAULT_TABLE table=test-table [id=faf72a0ecd904d25a34eb7f7913a2b99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:57.426180 26361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1b4d51234ba407dbe191c24bbe6c0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:57.427747 26295 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap starting.
I20260807 06:35:57.428076 26295 tablet_bootstrap.cc:654] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:57.428630 26295 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: No bootstrap required, opened a new log
I20260807 06:35:57.428674 26295 ts_tablet_manager.cc:1397] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.429044 26295 raft_consensus.cc:348] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.429116 26295 raft_consensus.cc:374] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:57.429152 26295 raft_consensus.cc:729] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f962797e14c919aa14f93a95b74f9, State: Initialized, Role: FOLLOWER
I20260807 06:35:57.429201 26295 consensus_queue.cc:260] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.429315 26295 ts_tablet_manager.cc:1428] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.429821 26874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.425584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:43205 (local address 127.0.0.1:42236)
0807 06:35:57.425771 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.425782 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.425856 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.426241 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.426242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.426244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:57.426288 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.426290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.427398 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.427399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.428651 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.428655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.429570 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.429571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.429632 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.429656 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.429692 (+ 36us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.429736 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":78,"thread_start_us":99,"threads_started":1}
I20260807 06:35:57.429951 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.425584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:41419 (local address 127.0.0.1:53868)
0807 06:35:57.425753 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.425765 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.425823 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.426130 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.426132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.426133 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:57.426178 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.426180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.427400 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.427402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.428668 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.428669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.429675 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.429677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.429759 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.429763 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.429814 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.429859 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":51}
I20260807 06:35:57.430199 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.425896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.2.243.2:43205)
0807 06:35:57.426073 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.426077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.426087 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:35:57.426125 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.426132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.426144 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.426230 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:57.426343 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.427277 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.428760 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.429465 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.429935 (+ 470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.430009 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.430039 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.430089 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260807 06:35:57.430308 26876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.425695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53868 (local address 127.2.243.3:41419)
0807 06:35:57.425940 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.425945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.425954 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:35:57.425991 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.425992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.426009 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.426255 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:57.426318 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.427273 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.428747 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.429573 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.430057 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.430154 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.430179 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.430222 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":91,"threads_started":1}
I20260807 06:35:57.430925 26571 tablet_service.cc:1467] Processing CreateTablet for tablet d1b4d51234ba407dbe191c24bbe6c0a7 (DEFAULT_TABLE table=test-table [id=faf72a0ecd904d25a34eb7f7913a2b99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:57.430968 26768 tablet_service.cc:1467] Processing CreateTablet for tablet d1b4d51234ba407dbe191c24bbe6c0a7 (DEFAULT_TABLE table=test-table [id=faf72a0ecd904d25a34eb7f7913a2b99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:35:57.431320 26571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1b4d51234ba407dbe191c24bbe6c0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:57.431345 26768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1b4d51234ba407dbe191c24bbe6c0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:35:57.434149 26880 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Bootstrap starting.
I20260807 06:35:57.434204 26882 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Bootstrap starting.
I20260807 06:35:57.434893 26880 tablet_bootstrap.cc:654] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:57.435057 26882 tablet_bootstrap.cc:654] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Neither blocks nor log segments found. Creating new log.
I20260807 06:35:57.435485 26880 log.cc:824] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:57.435724 26882 log.cc:824] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:57.436628 26880 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: No bootstrap required, opened a new log
I20260807 06:35:57.436635 26882 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: No bootstrap required, opened a new log
I20260807 06:35:57.436714 26880 ts_tablet_manager.cc:1397] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:35:57.436728 26882 ts_tablet_manager.cc:1397] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:35:57.438118 26880 raft_consensus.cc:348] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.438120 26882 raft_consensus.cc:348] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.438325 26882 raft_consensus.cc:374] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:57.438319 26880 raft_consensus.cc:374] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:35:57.438395 26880 raft_consensus.cc:729] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81dedc1957b7412fbefc4c2de268fa30, State: Initialized, Role: FOLLOWER
I20260807 06:35:57.438396 26882 raft_consensus.cc:729] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a626be8a41e547e288d58326a062e763, State: Initialized, Role: FOLLOWER
I20260807 06:35:57.438522 26882 consensus_queue.cc:260] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.438522 26880 consensus_queue.cc:260] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.438836 26882 ts_tablet_manager.cc:1428] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:35:57.438822 26864 heartbeater.cc:503] Master 127.2.243.62:45145 was elected leader, sending a full tablet report...
I20260807 06:35:57.438823 26880 ts_tablet_manager.cc:1428] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:35:57.438958 26674 heartbeater.cc:503] Master 127.2.243.62:45145 was elected leader, sending a full tablet report...
I20260807 06:35:57.446091 26475 raft_consensus.cc:482] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:35:57.446221 26475 raft_consensus.cc:504] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.446556 26475 leader_election.cc:288] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419), a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
I20260807 06:35:57.450537 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.446858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:50749 (local address 127.2.243.2:43205)
0807 06:35:57.446908 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.446910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.446918 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:35:57.446940 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.446941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.446957 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.447037 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:57.447246 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.447974 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.449558 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.450108 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.450182 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.450192 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.450426 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.450470 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:35:57.450531 26876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.446755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:43313 (local address 127.2.243.3:41419)
0807 06:35:57.446801 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.446803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.447013 (+ 210us) server_negotiation.cc:407] Connection header received
0807 06:35:57.447050 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.447051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.447061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.447105 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:35:57.447254 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.447974 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.449525 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.450099 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.450181 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.450191 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.450426 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.450470 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:35:57.450534 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.446705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:43205 (local address 127.2.243.1:50749)
0807 06:35:57.446765 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.446776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.446824 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.447051 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.447052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.447053 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:57.447154 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.447156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.448058 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.448059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.449426 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.449429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.450192 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.450192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.450274 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.450302 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.450367 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.450423 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":54}
I20260807 06:35:57.450549 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.446705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:41419 (local address 127.2.243.1:43313)
0807 06:35:57.446927 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.446940 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.446979 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.447122 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:57.447123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:57.447125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:35:57.447190 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.447192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.448058 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.448059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.449426 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:57.449429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.450192 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.450193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:57.450274 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.450303 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 06:35:57.450380 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:35:57.450423 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":162,"thread_start_us":100,"threads_started":1}
I20260807 06:35:57.450856 26604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1b4d51234ba407dbe191c24bbe6c0a7" candidate_uuid: "6b4f962797e14c919aa14f93a95b74f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a626be8a41e547e288d58326a062e763" is_pre_election: true
I20260807 06:35:57.450933 26805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1b4d51234ba407dbe191c24bbe6c0a7" candidate_uuid: "6b4f962797e14c919aa14f93a95b74f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81dedc1957b7412fbefc4c2de268fa30" is_pre_election: true
I20260807 06:35:57.451020 26604 raft_consensus.cc:2454] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f962797e14c919aa14f93a95b74f9 in term 0.
I20260807 06:35:57.451089 26805 raft_consensus.cc:2454] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f962797e14c919aa14f93a95b74f9 in term 0.
I20260807 06:35:57.451225 26290 leader_election.cc:302] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f962797e14c919aa14f93a95b74f9, a626be8a41e547e288d58326a062e763; no voters:
I20260807 06:35:57.451418 26475 raft_consensus.cc:2781] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:35:57.451462 26475 raft_consensus.cc:482] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:35:57.451478 26475 raft_consensus.cc:3037] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:57.452145 26475 raft_consensus.cc:504] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.452303 26475 leader_election.cc:288] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 1 election: Requested vote from peers 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419), a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
I20260807 06:35:57.452471 26805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1b4d51234ba407dbe191c24bbe6c0a7" candidate_uuid: "6b4f962797e14c919aa14f93a95b74f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81dedc1957b7412fbefc4c2de268fa30"
I20260807 06:35:57.452486 26604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1b4d51234ba407dbe191c24bbe6c0a7" candidate_uuid: "6b4f962797e14c919aa14f93a95b74f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a626be8a41e547e288d58326a062e763"
I20260807 06:35:57.452576 26805 raft_consensus.cc:3037] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:57.452576 26604 raft_consensus.cc:3037] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:35:57.453481 26604 raft_consensus.cc:2454] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f962797e14c919aa14f93a95b74f9 in term 1.
I20260807 06:35:57.453481 26805 raft_consensus.cc:2454] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f962797e14c919aa14f93a95b74f9 in term 1.
I20260807 06:35:57.453670 26288 leader_election.cc:302] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f962797e14c919aa14f93a95b74f9, 81dedc1957b7412fbefc4c2de268fa30; no voters:
I20260807 06:35:57.453786 26475 raft_consensus.cc:2781] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:35:57.453941 26475 raft_consensus.cc:686] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 LEADER]: Becoming Leader. State: Replica: 6b4f962797e14c919aa14f93a95b74f9, State: Running, Role: LEADER
I20260807 06:35:57.454061 26475 consensus_queue.cc:237] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.454871 26179 catalog_manager.cc:5495] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1). New cstate: current_term: 1 leader_uuid: "6b4f962797e14c919aa14f93a95b74f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } health_report { overall_health: HEALTHY } } }
I20260807 06:35:57.466682 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.464253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53258 (local address 127.2.243.1:33335)
0807 06:35:57.464311 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:57.464313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:57.464321 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:35:57.464358 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:57.464358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:57.464383 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:57.464458 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:35:57.464613 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.465179 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.465882 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:57.466111 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:57.466185 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:57.466355 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:35:57.466454 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:35:57.466537 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:57.466577 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 06:35:57.466604 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:35:57.468147 26604 raft_consensus.cc:1261] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Refusing update from remote peer 6b4f962797e14c919aa14f93a95b74f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:57.468166 26805 raft_consensus.cc:1261] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 FOLLOWER]: Refusing update from remote peer 6b4f962797e14c919aa14f93a95b74f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:35:57.468448 26475 consensus_queue.cc:1035] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:57.468601 26893 consensus_queue.cc:1035] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:35:57.471316 26897 mvcc.cc:205] Tried to move back new op lower bound from 7315802347387146240 to 7315802347332395008. Current Snapshot: MvccSnapshot[applied={T|T < 7315802347387146240}]
I20260807 06:35:57.473302 26179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:57.473395 26179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:35:57.474129 26179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Sending DeleteTablet for 3 replicas of tablet d1b4d51234ba407dbe191c24bbe6c0a7
I20260807 06:35:57.474450 26571 tablet_service.cc:1514] Processing DeleteTablet for tablet d1b4d51234ba407dbe191c24bbe6c0a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:57 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:42236
I20260807 06:35:57.474480 26361 tablet_service.cc:1514] Processing DeleteTablet for tablet d1b4d51234ba407dbe191c24bbe6c0a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:57 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:53246
I20260807 06:35:57.474514 26768 tablet_service.cc:1514] Processing DeleteTablet for tablet d1b4d51234ba407dbe191c24bbe6c0a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:35:57 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:53868
I20260807 06:35:57.474571 26483 tablet_replica.cc:330] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: stopping tablet replica
I20260807 06:35:57.474622 26483 raft_consensus.cc:2229] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:35:57.474711 26483 raft_consensus.cc:2258] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:57.474750 26906 tablet_replica.cc:330] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: stopping tablet replica
I20260807 06:35:57.474761 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 26234
I20260807 06:35:57.474839 26906 raft_consensus.cc:2229] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:57.484205 26908 tablet_replica.cc:330] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: stopping tablet replica
I20260807 06:35:57.484330 26908 raft_consensus.cc:2229] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:35:57.484443 26908 raft_consensus.cc:2258] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:57.485124 26908 ts_tablet_manager.cc:1905] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:35:57.485190 26906 raft_consensus.cc:2258] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:35:57.486901 26908 ts_tablet_manager.cc:1918] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:57.486958 26908 log.cc:1196] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/d1b4d51234ba407dbe191c24bbe6c0a7
I20260807 06:35:57.487267 26908 ts_tablet_manager.cc:1939] T d1b4d51234ba407dbe191c24bbe6c0a7 P 81dedc1957b7412fbefc4c2de268fa30: Deleting consensus metadata
I20260807 06:35:57.487672 26151 catalog_manager.cc:4841] TS 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419): tablet d1b4d51234ba407dbe191c24bbe6c0a7 (table test-table [id=faf72a0ecd904d25a34eb7f7913a2b99]) successfully deleted
I20260807 06:35:57.488941 26906 ts_tablet_manager.cc:1905] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:35:57.489786 26151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:33335 (error 108)
I20260807 06:35:57.490126 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 26474
I20260807 06:35:57.490278 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.490159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:33335 (local address 127.0.0.1:53274)
0807 06:35:57.490211 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.490244 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:33335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:35:57.499486 26151 catalog_manager.cc:4584] TS 6b4f962797e14c919aa14f93a95b74f9 (127.2.243.1:33335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1b4d51234ba407dbe191c24bbe6c0a7: Network error: Client connection negotiation failed: client connection to 127.2.243.1:33335: connect: Connection refused (error 111)
I20260807 06:35:57.517241 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 26679
I20260807 06:35:57.517297 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.517167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:43205 (local address 127.0.0.1:42248)
0807 06:35:57.517230 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.517262 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.2:43205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:35:57.538522 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 26120
I20260807 06:35:57.548769 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45145
--webserver_interface=127.2.243.62
--webserver_port=44939
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:45145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:57.629988 26913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:57.630273 26913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:57.630303 26913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:57.631704 26913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:35:57.631771 26913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:57.631788 26913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:57.631800 26913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:35:57.631814 26913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:35:57.633430 26913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:45145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:45145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:57.633658 26913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:57.633865 26913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:35:57.639137 26913 server_base.cc:1034] running on GCE node
W20260807 06:35:57.638984 26926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.638984 26922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.638993 26923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.639449 26913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:57.639808 26913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:57.641045 26913 hybrid_clock.cc:648] HybridClock initialized: now 1786084557641034 us; error 40 us; skew 500 ppm
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:57.643756 26913 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:35:57.644232 26913 webserver.cc:460] Webserver started at http://127.2.243.62:44939/ using document root <none> and password file <none>
I20260807 06:35:57.644446 26913 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:57.644529 26913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:57.645848 26913 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.646493 26934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.646690 26913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.646786 26913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "87a5b9be1263440e89df2a5861ad0225"
format_stamp: "Formatted at 2026-08-07 06:35:55 on dist-test-slave-rwrg"
server_key: "b392ea61417d268beac803f8e5606768"
server_key_iv: "f600c28bd02de1455b9a6768797f2059"
server_key_version: "encryptionkey@0"
I20260807 06:35:57.647109 26913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:57.670804 26913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:57.672803 26913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:57.672920 26913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:57.676796 26913 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:45145
I20260807 06:35:57.676868 27009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:45145 every 8 connection(s)
I20260807 06:35:57.677287 26913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:35:57.678880 27012 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:35:57.679531 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Bootstrap starting.
I20260807 06:35:57.684384 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 26913
I20260807 06:35:57.684715 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:33335
--local_ip_for_outbound_sockets=127.2.243.1
--tserver_master_addrs=127.2.243.62:45145
--webserver_port=36223
--webserver_interface=127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:35:57.688314 27012 log.cc:824] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:57.690711 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:35:57.691027 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225: Bootstrap complete.
I20260807 06:35:57.693346 27012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.693585 27012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a5b9be1263440e89df2a5861ad0225, State: Initialized, Role: FOLLOWER
I20260807 06:35:57.693764 27012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.693840 27012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:35:57.693889 27012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:35:57.693938 27012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:35:57.694932 27012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.695067 27012 leader_election.cc:302] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87a5b9be1263440e89df2a5861ad0225; no voters:
I20260807 06:35:57.695281 27012 leader_election.cc:288] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 06:35:57.695317 27022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:35:57.695431 27022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [term 3 LEADER]: Becoming Leader. State: Replica: 87a5b9be1263440e89df2a5861ad0225, State: Running, Role: LEADER
I20260807 06:35:57.695542 27022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } }
I20260807 06:35:57.695649 27012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:35:57.695797 27022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87a5b9be1263440e89df2a5861ad0225. Latest consensus state: current_term: 3 leader_uuid: "87a5b9be1263440e89df2a5861ad0225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } } }
I20260807 06:35:57.695878 27022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:57.695802 27024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "87a5b9be1263440e89df2a5861ad0225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a5b9be1263440e89df2a5861ad0225" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 45145 } } }
I20260807 06:35:57.696087 27024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a5b9be1263440e89df2a5861ad0225 [sys.catalog]: This master's current role is: LEADER
I20260807 06:35:57.697005 27031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:35:57.697782 27031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:35:57.697885 27031 catalog_manager.cc:1259] Loaded cluster ID: 5207e1fde2984fcd98dd980231cb74d4
I20260807 06:35:57.697916 27031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 06:35:57.698215 27031 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f13a5be59ad google::LogMessage::Fail() at ??:0
@ 0x7f13a5be888a google::LogMessage::SendToLog() at ??:0
@ 0x7f13a5be54de google::LogMessage::Flush() at ??:0
@ 0x7f13a5be6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f13adb060f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f13adb0f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f13a7ab30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f13a7aa8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f13a890c6db start_thread at ??:0
@ 0x7f13a46e361f clone at ??:0
W20260807 06:35:57.786016 27016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:57.786231 27016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:57.786257 27016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:57.787432 27016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:57.787477 27016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:57.787504 27016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:35:57.789093 27016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:33335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=36223
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:57.789366 27016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:57.789624 27016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:57.795812 27077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.795950 27073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.795836 27074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.795987 27016 server_base.cc:1034] running on GCE node
I20260807 06:35:57.796269 27016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:57.796571 27016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:57.797747 27016 hybrid_clock.cc:648] HybridClock initialized: now 1786084557797729 us; error 36 us; skew 500 ppm
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:57.800364 27016 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:35:57.800935 27016 webserver.cc:460] Webserver started at http://127.2.243.1:36223/ using document root <none> and password file <none>
I20260807 06:35:57.801203 27016 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:57.801271 27016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:57.802719 27016 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.803465 27089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.803655 27016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.803764 27016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "6b4f962797e14c919aa14f93a95b74f9"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "48d28b491911ef57a24495a8316f504e"
server_key_iv: "9d9fa8bbc9956b80ea00c48202f9b77e"
server_key_version: "encryptionkey@0"
I20260807 06:35:57.804119 27016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:57.811928 27016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:57.814230 27016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:57.814370 27016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:57.814605 27016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:57.815092 27097 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:35:57.817087 27016 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:35:57.817174 27016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:35:57.817224 27016 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:35:57.817850 27016 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:35:57.817909 27016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:35:57.817994 27097 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap starting.
I20260807 06:35:57.825904 27016 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:33335
I20260807 06:35:57.826481 27016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:35:57.828658 27097 log.cc:824] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:35:57.829408 27265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:33335 every 8 connection(s)
I20260807 06:35:57.831316 27097 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:35:57.831629 27097 tablet_bootstrap.cc:492] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Bootstrap complete.
I20260807 06:35:57.832646 27097 ts_tablet_manager.cc:1397] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.010s
I20260807 06:35:57.832739 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 27016
I20260807 06:35:57.833003 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:43205
--local_ip_for_outbound_sockets=127.2.243.2
--tserver_master_addrs=127.2.243.62:45145
--webserver_port=46297
--webserver_interface=127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:35:57.834384 27097 raft_consensus.cc:348] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.834789 27097 raft_consensus.cc:729] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f962797e14c919aa14f93a95b74f9, State: Initialized, Role: FOLLOWER
I20260807 06:35:57.834985 27097 consensus_queue.cc:260] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:57.835245 27097 ts_tablet_manager.cc:1428] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260807 06:35:57.905934 27270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:35:57.830038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:49857)
0807 06:35:57.830239 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.830287 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:57.831470 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:57.905747 (+ 74277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
W20260807 06:35:57.906232 27266 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:45145 (0 consecutive failures): Network error: Failed to ping master at 127.2.243.62:45145: Client connection negotiation failed: client connection to 127.2.243.62:45145: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:35:57.906587 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.906468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:46095)
0807 06:35:57.906527 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.906568 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 06:35:57.906949 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.906869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:36497)
0807 06:35:57.906912 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.906935 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 06:35:57.907083 27266 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:35:57.928416 27271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:57.928753 27271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:57.928781 27271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:57.930090 27271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:57.930128 27271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:57.930155 27271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:35:57.931717 27271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:43205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=46297
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:57.931952 27271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:57.932178 27271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:57.937247 27282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.937270 27286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:57.937247 27284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:57.937490 27271 server_base.cc:1034] running on GCE node
I20260807 06:35:57.937690 27271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:57.937889 27271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:57.939057 27271 hybrid_clock.cc:648] HybridClock initialized: now 1786084557939045 us; error 29 us; skew 500 ppm
Aug 07 06:35:57 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:57.941622 27271 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:35:57.942042 27271 webserver.cc:460] Webserver started at http://127.2.243.2:46297/ using document root <none> and password file <none>
I20260807 06:35:57.942258 27271 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:57.942303 27271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:57.943635 27271 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:35:57.944312 27301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.944473 27271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:35:57.944643 27271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "a626be8a41e547e288d58326a062e763"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "1682a608b9d787056e9889edaa5b250c"
server_key_iv: "f63e5f2fc658bd9b4cef660a20b7986c"
server_key_version: "encryptionkey@0"
I20260807 06:35:57.945000 27271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:57.958145 27271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:57.960178 27271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:57.960337 27271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:57.960680 27271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:57.961170 27310 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260807 06:35:57.962604 27310 ts_tablet_manager.cc:1852] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 06:35:57.962868 27310 ts_tablet_manager.cc:1905] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:35:57.964948 27310 ts_tablet_manager.cc:1918] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:35:57.965036 27310 log.cc:1196] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/d1b4d51234ba407dbe191c24bbe6c0a7
I20260807 06:35:57.965339 27310 ts_tablet_manager.cc:1939] T d1b4d51234ba407dbe191c24bbe6c0a7 P a626be8a41e547e288d58326a062e763: Deleting consensus metadata
I20260807 06:35:57.965559 27271 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 06:35:57.965721 27271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 06:35:57.965782 27271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:57.965814 27271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:57.971716 27271 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:43205
I20260807 06:35:57.971755 27474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:43205 every 8 connection(s)
I20260807 06:35:57.972065 27271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:35:57.973177 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.972815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:50969)
0807 06:35:57.972981 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.973051 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
W20260807 06:35:57.973405 27475 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:45145 (0 consecutive failures): Network error: Failed to ping master at 127.2.243.62:45145: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
I20260807 06:35:57.973776 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.973665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:43877)
0807 06:35:57.973728 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.973757 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:35:57.974123 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:57.974041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:47013)
0807 06:35:57.974083 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:57.974106 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 06:35:57.974247 27475 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:35:57.981890 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 27271
I20260807 06:35:57.982200 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:41419
--local_ip_for_outbound_sockets=127.2.243.3
--tserver_master_addrs=127.2.243.62:45145
--webserver_port=34889
--webserver_interface=127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:35:58.064997 27482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:35:58.065217 27482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:35:58.065241 27482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:35:58.066347 27482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:35:58.066385 27482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:35:58.066411 27482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:35:58.067811 27482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:41419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=34889
--tserver_master_addrs=127.2.243.62:45145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:35:58.068006 27482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:35:58.068174 27482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:35:58.073335 27500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:58.073356 27491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:35:58.073333 27493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:35:58.073563 27482 server_base.cc:1034] running on GCE node
I20260807 06:35:58.073865 27482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:35:58.074139 27482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:35:58.075326 27482 hybrid_clock.cc:648] HybridClock initialized: now 1786084558075321 us; error 39 us; skew 500 ppm
Aug 07 06:35:58 dist-test-slave-rwrg krb5kdc[25265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084558, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:35:58.077970 27482 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:35:58.078460 27482 webserver.cc:460] Webserver started at http://127.2.243.3:34889/ using document root <none> and password file <none>
I20260807 06:35:58.078701 27482 fs_manager.cc:359] Metadata directory not provided
I20260807 06:35:58.078747 27482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:35:58.080156 27482 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:58.080848 27522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:58.081037 27482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:35:58.081128 27482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "81dedc1957b7412fbefc4c2de268fa30"
format_stamp: "Formatted at 2026-08-07 06:35:56 on dist-test-slave-rwrg"
server_key: "30c583f0ab94f42fd218e21921269c58"
server_key_iv: "f20cf8d3592c7db02a8e2b773deebaed"
server_key_version: "encryptionkey@0"
I20260807 06:35:58.081506 27482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:35:58.088616 27482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:35:58.090359 27482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:35:58.090516 27482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:35:58.090727 27482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:35:58.091032 27482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:35:58.091064 27482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:58.091084 27482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:35:58.091130 27482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:35:58.096963 27482 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:41419
I20260807 06:35:58.097014 27689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:41419 every 8 connection(s)
I20260807 06:35:58.097332 27482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:35:58.097929 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 27482
I20260807 06:35:58.098376 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:58.097941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:55425)
0807 06:35:58.098122 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:58.098215 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
W20260807 06:35:58.098807 27692 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:45145 (0 consecutive failures): Network error: Failed to ping master at 127.2.243.62:45145: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
I20260807 06:35:58.099069 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:58.098980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:54951)
0807 06:35:58.099017 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:58.099050 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:35:58.099334 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:58.099261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:41283)
0807 06:35:58.099299 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:58.099320 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 06:35:58.099455 27692 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:35:58.907752 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:58.907430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:42909)
0807 06:35:58.907593 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:58.907669 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
I20260807 06:35:58.974743 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:58.974555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:57597)
0807 06:35:58.974675 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:58.974709 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260807 06:35:59.099962 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:59.099765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:33367)
0807 06:35:59.099880 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:59.099911 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260807 06:35:59.129700 27704 raft_consensus.cc:482] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:35:59.129915 27704 raft_consensus.cc:504] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:35:59.130292 27704 leader_election.cc:288] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419), a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
Aug 07 06:35:59 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.3@KRBTEST.COM
Aug 07 06:35:59 dist-test-slave-rwrg krb5kdc[25265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084557, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.2@KRBTEST.COM
I20260807 06:35:59.143639 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:59.130602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:38893 (local address 127.2.243.3:41419)
0807 06:35:59.130788 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:59.130803 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:59.130837 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:35:59.131461 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:59.131465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:59.131534 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:59.131693 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:59.132757 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.134165 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.135779 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.136461 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.136631 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:59.139593 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:59.139599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:59.139599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:59.139613 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:59.141783 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:59.142344 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:59.142348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:59.142349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:59.142461 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:59.142758 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:59.142759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:59.142760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:59.142991 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:59.143098 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:59.143391 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 06:35:59.143518 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":66,"threads_started":1}
I20260807 06:35:59.143652 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:59.130578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:46555 (local address 127.2.243.2:43205)
0807 06:35:59.130803 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 06:35:59.130810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:35:59.130833 (+ 23us) server_negotiation.cc:407] Connection header received
0807 06:35:59.131474 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:35:59.131477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:35:59.131531 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:35:59.131693 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:35:59.132758 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.134163 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.135779 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.136461 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.136632 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:59.140093 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:35:59.140097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:35:59.140098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:35:59.140110 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:35:59.141943 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:59.142372 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:59.142374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:59.142374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:59.142458 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:35:59.142729 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:35:59.142730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:35:59.142730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:35:59.142988 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:35:59.143082 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:35:59.143378 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 06:35:59.143512 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260807 06:35:59.143682 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:59.130442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:43205 (local address 127.2.243.1:46555)
0807 06:35:59.130496 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:59.130512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:59.130661 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:59.131790 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:59.131793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:59.132298 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:59.132621 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:59.132629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.134305 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.134305 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:59.135630 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:59.135634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.136612 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.136614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:59.136777 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:59.137417 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:59.137429 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:59.139894 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:59.142045 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:59.142046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:59.142047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:59.142264 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:59.142538 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:59.142539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:59.142540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:59.142660 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:59.143070 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:59.143078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:59.143216 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 06:35:59.143321 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:35:59.143570 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":91}
I20260807 06:35:59.144198 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:59.130442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:41419 (local address 127.2.243.1:38893)
0807 06:35:59.130590 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:59.130603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:35:59.130690 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:35:59.131790 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:35:59.131793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:35:59.132298 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:35:59.132621 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:59.132629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.134293 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.134294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:59.135630 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:35:59.135634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:35:59.136614 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:35:59.136615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:35:59.136777 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:35:59.137419 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:35:59.137429 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:35:59.139449 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:35:59.141881 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:59.141889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:59.141892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:59.142252 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:59.142542 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:35:59.142543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:35:59.142544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:35:59.142669 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:35:59.143091 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:35:59.143092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:35:59.143213 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 06:35:59.143320 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:35:59.143622 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":206,"thread_start_us":62,"threads_started":1}
W20260807 06:35:59.144658 27093 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a626be8a41e547e288d58326a062e763 (127.2.243.2:43205): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
W20260807 06:35:59.144923 27090 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
I20260807 06:35:59.144996 27090 leader_election.cc:302] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b4f962797e14c919aa14f93a95b74f9; no voters: 81dedc1957b7412fbefc4c2de268fa30, a626be8a41e547e288d58326a062e763
I20260807 06:35:59.145118 27704 raft_consensus.cc:2726] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:35:59.908420 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:59.908208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:54407)
0807 06:35:59.908337 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:59.908374 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260807 06:35:59.975445 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:35:59.975259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:54903)
0807 06:35:59.975375 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 06:35:59.975409 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260807 06:36:00.100667 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:00.100429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:56581)
0807 06:36:00.100577 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:00.100616 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260807 06:36:00.690071 27714 raft_consensus.cc:482] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:36:00.690172 27714 raft_consensus.cc:504] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:36:00.690358 27714 leader_election.cc:288] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419), a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
W20260807 06:36:00.690938 27093 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a626be8a41e547e288d58326a062e763 (127.2.243.2:43205): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
W20260807 06:36:00.691052 27090 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
I20260807 06:36:00.691109 27090 leader_election.cc:302] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b4f962797e14c919aa14f93a95b74f9; no voters: 81dedc1957b7412fbefc4c2de268fa30, a626be8a41e547e288d58326a062e763
I20260807 06:36:00.691193 27714 raft_consensus.cc:2726] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:36:00.909276 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:00.909042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:51837)
0807 06:36:00.909179 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:00.909219 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260807 06:36:00.976155 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:00.975947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:45527)
0807 06:36:00.976082 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:00.976119 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260807 06:36:01.101438 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:01.101193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:36495)
0807 06:36:01.101348 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:01.101396 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260807 06:36:01.909934 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:01.909714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:38027)
0807 06:36:01.909849 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:01.909886 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260807 06:36:01.977028 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:01.976754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:36949)
0807 06:36:01.976908 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:01.976966 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260807 06:36:02.102128 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:02.101926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:54181)
0807 06:36:02.102047 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:02.102085 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260807 06:36:02.910662 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:02.910433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:35977)
0807 06:36:02.910578 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:02.910618 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260807 06:36:02.977707 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:02.977494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:50847)
0807 06:36:02.977624 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:02.977663 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260807 06:36:03.102866 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:03.102629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:47163)
0807 06:36:03.102790 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:03.102826 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260807 06:36:03.212101 27727 raft_consensus.cc:482] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:36:03.212199 27727 raft_consensus.cc:504] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:36:03.212397 27727 leader_election.cc:288] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419), a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
W20260807 06:36:03.212870 27093 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a626be8a41e547e288d58326a062e763 (127.2.243.2:43205): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
W20260807 06:36:03.212972 27090 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
I20260807 06:36:03.213085 27090 leader_election.cc:302] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b4f962797e14c919aa14f93a95b74f9; no voters: 81dedc1957b7412fbefc4c2de268fa30, a626be8a41e547e288d58326a062e763
I20260807 06:36:03.213204 27727 raft_consensus.cc:2726] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:36:03.911572 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:03.911288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:38593)
0807 06:36:03.911463 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:03.911519 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260807 06:36:03.978457 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:03.978231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:50193)
0807 06:36:03.978384 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:03.978413 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260807 06:36:04.103561 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:04.103362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:46785)
0807 06:36:04.103486 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:04.103519 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260807 06:36:04.912415 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:04.912203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:35913)
0807 06:36:04.912340 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:04.912370 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260807 06:36:04.979259 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:04.979022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:41047)
0807 06:36:04.979167 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:04.979206 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":60,"threads_started":1}
I20260807 06:36:05.104265 27741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:05.104041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:46033)
0807 06:36:05.104185 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:05.104223 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260807 06:36:05.858969 27744 raft_consensus.cc:482] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:36:05.859092 27744 raft_consensus.cc:504] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81dedc1957b7412fbefc4c2de268fa30" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 41419 } } peers { permanent_uuid: "a626be8a41e547e288d58326a062e763" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 43205 } } peers { permanent_uuid: "6b4f962797e14c919aa14f93a95b74f9" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 33335 } }
I20260807 06:36:05.859275 27744 leader_election.cc:288] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419), a626be8a41e547e288d58326a062e763 (127.2.243.2:43205)
W20260807 06:36:05.859794 27090 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81dedc1957b7412fbefc4c2de268fa30 (127.2.243.3:41419): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
W20260807 06:36:05.859915 27093 leader_election.cc:341] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a626be8a41e547e288d58326a062e763 (127.2.243.2:43205): Not found: Tablet not found: d1b4d51234ba407dbe191c24bbe6c0a7
I20260807 06:36:05.859988 27093 leader_election.cc:302] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b4f962797e14c919aa14f93a95b74f9; no voters: 81dedc1957b7412fbefc4c2de268fa30, a626be8a41e547e288d58326a062e763
I20260807 06:36:05.860124 27744 raft_consensus.cc:2726] T d1b4d51234ba407dbe191c24bbe6c0a7 P 6b4f962797e14c919aa14f93a95b74f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:36:05.913153 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:05.912890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:46615)
0807 06:36:05.913051 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:05.913106 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260807 06:36:05.979995 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:05.979714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:35893)
0807 06:36:05.979897 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:05.979941 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
I20260807 06:36:06.104946 27747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:06.104734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:59049)
0807 06:36:06.104853 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:06.104889 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260807 06:36:06.914153 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:06.913807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:47863)
0807 06:36:06.914001 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:06.914083 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":125,"threads_started":1}
I20260807 06:36:06.981015 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:06.980646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:46771)
0807 06:36:06.980863 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:06.980933 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":129,"threads_started":1}
I20260807 06:36:07.105921 27750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:07.105502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:46561)
0807 06:36:07.105758 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:07.105838 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":131,"threads_started":1}
I20260807 06:36:07.915254 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:07.914900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:33215)
0807 06:36:07.915164 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:07.915208 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":117,"threads_started":1}
I20260807 06:36:07.981860 27752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:07.981634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:56001)
0807 06:36:07.981779 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:07.981818 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260807 06:36:08.106870 27754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:08.106574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:37105)
0807 06:36:08.106709 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:08.106808 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260807 06:36:08.915946 27755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:08.915742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:52133)
0807 06:36:08.915866 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:08.915900 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260807 06:36:08.982517 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:08.982300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:58309)
0807 06:36:08.982436 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:08.982474 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260807 06:36:09.107703 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:09.107377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:54803)
0807 06:36:09.107591 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:09.107658 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":71,"threads_started":1}
I20260807 06:36:09.916710 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:09.916410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:38371)
0807 06:36:09.916617 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:09.916662 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":67,"threads_started":1}
I20260807 06:36:09.983180 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:09.982976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:58563)
0807 06:36:09.983100 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:09.983137 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260807 06:36:10.108363 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:10.108152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:53747)
0807 06:36:10.108292 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:10.108323 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260807 06:36:10.917471 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:10.917245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:51325)
0807 06:36:10.917388 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:10.917426 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260807 06:36:10.983911 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:10.983647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:38491)
0807 06:36:10.983822 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:10.983866 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260807 06:36:11.109103 27766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:11.108883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:47159)
0807 06:36:11.109025 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:11.109059 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260807 06:36:11.918247 27767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:11.917905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:47075)
0807 06:36:11.918133 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:11.918178 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":100,"threads_started":1}
I20260807 06:36:11.984669 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:11.984385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:58059)
0807 06:36:11.984573 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:11.984621 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260807 06:36:12.109799 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:12.109585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:53937)
0807 06:36:12.109717 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:12.109754 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260807 06:36:12.919121 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:12.918872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.1:53023)
0807 06:36:12.919029 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:12.919075 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260807 06:36:12.985363 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:12.985156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.2:36169)
0807 06:36:12.985290 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:12.985321 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260807 06:36:13.100137 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 27016
I20260807 06:36:13.110556 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:13.110310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:45145 (local address 127.2.243.3:55727)
0807 06:36:13.110428 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:13.110477 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:45145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260807 06:36:13.114527 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 27271
I20260807 06:36:13.124485 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 27482
2026-08-07T06:36:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18628 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:36:13 dist-test-slave-rwrg krb5kdc[27784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:36:13 dist-test-slave-rwrg krb5kdc[27784](info): set up 2 sockets
Aug 07 06:36:13 dist-test-slave-rwrg krb5kdc[27784](info): commencing operation
krb5kdc: starting...
W20260807 06:36:14.401607 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:36:14 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:36:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:36:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:36:14.523450 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39945
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:36:14.607026 27807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:14.607246 27807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:14.607270 27807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:14.608736 27807 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:36:14.608783 27807 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:36:14.608795 27807 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:36:14.608807 27807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:36:14.608820 27807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:14.608834 27807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:14.608845 27807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:36:14.608857 27807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:36:14.610716 27807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:14.610913 27807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:14.611079 27807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:14.616408 27819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:14.616452 27813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:14.616410 27816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:14.616657 27807 server_base.cc:1034] running on GCE node
I20260807 06:36:14.617025 27807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:14.617301 27807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:14.618443 27807 hybrid_clock.cc:648] HybridClock initialized: now 1786084574618433 us; error 36 us; skew 500 ppm
Aug 07 06:36:14 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084574, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:14.621130 27807 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:36:14.621620 27807 webserver.cc:460] Webserver started at http://127.2.243.62:40913/ using document root <none> and password file <none>
I20260807 06:36:14.621846 27807 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:14.621891 27807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:14.622012 27807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:36:14.623102 27807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "092291564fcc464d83fd4035f6b59070"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "64d1e821bef22195cecf5ad6a43965f2"
server_key_iv: "e814c36603662e112cfa7a5182a76ef8"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.623456 27807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "092291564fcc464d83fd4035f6b59070"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "64d1e821bef22195cecf5ad6a43965f2"
server_key_iv: "e814c36603662e112cfa7a5182a76ef8"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.624830 27807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:36:14.625620 27833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:14.625811 27807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:14.625931 27807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "092291564fcc464d83fd4035f6b59070"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "64d1e821bef22195cecf5ad6a43965f2"
server_key_iv: "e814c36603662e112cfa7a5182a76ef8"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.625999 27807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:14.642860 27807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:14.645148 27807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:14.645311 27807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:14.649497 27807 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39945
I20260807 06:36:14.649544 27914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39945 every 8 connection(s)
I20260807 06:36:14.649906 27807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:36:14.650789 27918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:36:14.653153 27918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Bootstrap starting.
I20260807 06:36:14.653785 27918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Neither blocks nor log segments found. Creating new log.
I20260807 06:36:14.654094 27918 log.cc:824] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:14.654832 27918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: No bootstrap required, opened a new log
I20260807 06:36:14.656389 27918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:14.656582 27918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:36:14.656628 27918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 092291564fcc464d83fd4035f6b59070, State: Initialized, Role: FOLLOWER
I20260807 06:36:14.656770 27918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:14.656826 27918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:36:14.656845 27918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:36:14.656873 27918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:36:14.657404 27918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:14.657510 27918 leader_election.cc:302] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 092291564fcc464d83fd4035f6b59070; no voters:
I20260807 06:36:14.657672 27918 leader_election.cc:288] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:36:14.657748 27922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:36:14.657931 27922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 LEADER]: Becoming Leader. State: Replica: 092291564fcc464d83fd4035f6b59070, State: Running, Role: LEADER
I20260807 06:36:14.657975 27918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:36:14.658321 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 27807
I20260807 06:36:14.658440 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:36:14.658105 27922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:14.658694 3020 external_mini_cluster.cc:1428] Setting key 4efbc20b94d80bbfe4e570fc8e134fd8
I20260807 06:36:14.659092 27923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "092291564fcc464d83fd4035f6b59070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } } }
I20260807 06:36:14.659180 27923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: This master's current role is: LEADER
I20260807 06:36:14.659462 27924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 092291564fcc464d83fd4035f6b59070. Latest consensus state: current_term: 1 leader_uuid: "092291564fcc464d83fd4035f6b59070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } } }
I20260807 06:36:14.659642 27924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: This master's current role is: LEADER
W20260807 06:36:14.660146 27946 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:36:14.660262 27946 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:36:14.660375 27941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:36:14.660578 27941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:36:14.662106 27941 catalog_manager.cc:1347] Generated new cluster ID: 4c958c67d0aa474f9c3ef3f7a843114a
I20260807 06:36:14.662158 27941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:36:14 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:36:14.670665 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:14.659176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33346 (local address 127.2.243.62:39945)
0807 06:36:14.659657 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:14.659667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:14.659702 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:36:14.659943 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:14.659950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:14.660008 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:14.660177 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:14.661024 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:14.662256 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:14.663325 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:14.663925 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:14.664023 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:14.666891 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:14.666896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:14.666896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:14.666911 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:14.668962 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:14.669361 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:14.669365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:14.669365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:14.669478 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:14.669686 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:14.669687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:14.669688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:14.670053 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:14.670201 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:14.670301 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 06:36:14.670452 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":98,"threads_started":1}
I20260807 06:36:14.689198 27941 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:36:14.689970 27941 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:36:14.700888 27941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Generated new TSK 0
I20260807 06:36:14.701079 27941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:36:14.731482 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39945
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:36:14.814118 27957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:14.814313 27957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:14.814335 27957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:14.815429 27957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:14.815467 27957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:14.815493 27957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:36:14.816955 27957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:14.817128 27957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:14.817301 27957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:14.821955 27971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:14.821974 27967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:14.821996 27969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:14.822093 27957 server_base.cc:1034] running on GCE node
I20260807 06:36:14.822388 27957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:14.822579 27957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:14.823707 27957 hybrid_clock.cc:648] HybridClock initialized: now 1786084574823697 us; error 27 us; skew 500 ppm
Aug 07 06:36:14 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084574, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:14.826182 27957 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:36:14.826685 27957 webserver.cc:460] Webserver started at http://127.2.243.1:45259/ using document root <none> and password file <none>
I20260807 06:36:14.826915 27957 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:14.826978 27957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:14.827126 27957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:36:14.828138 27957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "a503483943f74de19ef8cb43cbb08d80"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "e4bf08fa5b6708a0c1b462719c67f481"
server_key_iv: "fa41f593499ae376e7e5418f9c9bc16a"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.828590 27957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "a503483943f74de19ef8cb43cbb08d80"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "e4bf08fa5b6708a0c1b462719c67f481"
server_key_iv: "fa41f593499ae376e7e5418f9c9bc16a"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.831992 27957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 06:36:14.832801 27980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:14.832967 27957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:36:14.833068 27957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "a503483943f74de19ef8cb43cbb08d80"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "e4bf08fa5b6708a0c1b462719c67f481"
server_key_iv: "fa41f593499ae376e7e5418f9c9bc16a"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.833148 27957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:14.847834 27957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:14.849946 27957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:14.850108 27957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:14.850332 27957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:14.850643 27957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:36:14.850693 27957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:14.850751 27957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:36:14.850780 27957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:14.856376 27957 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40043
I20260807 06:36:14.856455 28164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40043 every 8 connection(s)
I20260807 06:36:14.856781 27957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:36:14.856981 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 27957
I20260807 06:36:14.857110 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:36:14.857323 3020 external_mini_cluster.cc:1428] Setting key ce9522d0714d228aeb9e485bb64ddeab
Aug 07 06:36:14 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084574, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
I20260807 06:36:14.867050 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:14.857494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:34841 (local address 127.2.243.62:39945)
0807 06:36:14.857550 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:14.857552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:14.857837 (+ 285us) server_negotiation.cc:407] Connection header received
0807 06:36:14.858535 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:14.858536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:14.858547 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:14.858605 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:14.859974 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:14.860454 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:14.861371 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:14.861577 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:14.861639 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:14.863922 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:14.863923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:14.863924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:14.863936 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:14.865547 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:14.866002 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:14.866004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:14.866004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:14.866072 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:14.866352 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:14.866353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:14.866354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:14.866479 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:14.866565 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:14.866854 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 06:36:14.866944 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 06:36:14.867210 28167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:14.857475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:34841)
0807 06:36:14.857705 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:14.857748 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:14.858428 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:14.858675 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:14.858679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:14.859575 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:14.859854 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:14.859862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:14.860592 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:14.860593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:14.861302 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:14.861303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:14.861666 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:14.861667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:14.861808 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:14.862269 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:14.862278 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:14.863815 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:14.865661 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:14.865669 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:14.865671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:14.865911 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:14.866145 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:14.866146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:14.866146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:14.866263 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:14.866554 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:14.866561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:14.866705 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 06:36:14.866829 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:36:14.866984 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260807 06:36:14.867712 28165 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39945
I20260807 06:36:14.867806 28165 heartbeater.cc:464] Registering TS with master...
I20260807 06:36:14.867997 28165 heartbeater.cc:511] Master 127.2.243.62:39945 requested a full tablet report, sending...
I20260807 06:36:14.868604 27856 ts_manager.cc:194] Registered new tserver with Master: a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043)
I20260807 06:36:14.869189 27856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:34841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:36:14.886723 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39945
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:36:14.970387 28174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:14.970597 28174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:14.970621 28174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:14.971794 28174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:14.971835 28174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:14.971874 28174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:36:14.973371 28174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:14.973611 28174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:14.973845 28174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:14.978734 28183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:14.978780 28185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:14.978797 28174 server_base.cc:1034] running on GCE node
W20260807 06:36:14.978734 28188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:14.979144 28174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:14.979326 28174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:14.980461 28174 hybrid_clock.cc:648] HybridClock initialized: now 1786084574980448 us; error 29 us; skew 500 ppm
Aug 07 06:36:14 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084574, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:14.982928 28174 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:36:14.983371 28174 webserver.cc:460] Webserver started at http://127.2.243.2:44509/ using document root <none> and password file <none>
I20260807 06:36:14.983592 28174 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:14.983636 28174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:14.983777 28174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:36:14.984905 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "ee05bb2937cea15b8786ee591422dd08"
server_key_iv: "c3dae8318c35c3b13f92ad9a3dd07290"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.985258 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "ee05bb2937cea15b8786ee591422dd08"
server_key_iv: "c3dae8318c35c3b13f92ad9a3dd07290"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.986527 28174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:36:14.987190 28199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:14.987361 28174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:36:14.987452 28174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "ee05bb2937cea15b8786ee591422dd08"
server_key_iv: "c3dae8318c35c3b13f92ad9a3dd07290"
server_key_version: "encryptionkey@0"
I20260807 06:36:14.987522 28174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:14.998309 28174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:15.000051 28174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:15.000197 28174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:15.000415 28174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:15.000748 28174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:36:15.000808 28174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:15.000842 28174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:36:15.000869 28174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:15.006677 28174 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:34349
I20260807 06:36:15.006722 28379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:34349 every 8 connection(s)
I20260807 06:36:15.007046 28174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:36:15.011744 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 28174
I20260807 06:36:15.011873 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:36:15.012166 3020 external_mini_cluster.cc:1428] Setting key c42f91031de48b71adacc4733e08f722
Aug 07 06:36:15 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084574, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:36:15.016928 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.007747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:60135)
0807 06:36:15.007963 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.007998 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:15.008540 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:15.008782 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:15.008785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:15.009384 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:15.009596 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.009601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.010290 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.010291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.011074 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.011076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.011425 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.011426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.011529 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.012133 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:15.012144 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:15.013749 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:15.015447 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:15.015454 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:15.015456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:15.015777 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:15.015985 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:15.015986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:15.015986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:15.016094 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:15.016376 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:15.016380 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:15.016486 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 06:36:15.016660 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 06:36:15.016776 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260807 06:36:15.016938 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.007838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:60135 (local address 127.2.243.62:39945)
0807 06:36:15.007926 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.007928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.008091 (+ 163us) server_negotiation.cc:407] Connection header received
0807 06:36:15.008669 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.008670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.008681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.008722 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:15.009715 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.010205 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.011170 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.011353 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.011408 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.013836 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:15.013838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:15.013838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:15.013846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:15.015367 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:15.015855 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:15.015856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:15.015856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:15.015918 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:15.016156 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:15.016157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:15.016158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:15.016310 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:15.016384 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.016699 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.016798 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:36:15.017643 28380 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39945
I20260807 06:36:15.017753 28380 heartbeater.cc:464] Registering TS with master...
I20260807 06:36:15.017961 28380 heartbeater.cc:511] Master 127.2.243.62:39945 requested a full tablet report, sending...
I20260807 06:36:15.018350 27856 ts_manager.cc:194] Registered new tserver with Master: b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349)
I20260807 06:36:15.018711 27856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:60135
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:36:15.039515 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39945
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:36:15.120247 28389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:15.120437 28389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:15.120460 28389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:15.121582 28389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:15.121625 28389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:15.121678 28389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:36:15.123098 28389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:15.123327 28389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:15.123543 28389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:15.128276 28399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:15.128325 28397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:15.128326 28396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:15.128831 28389 server_base.cc:1034] running on GCE node
I20260807 06:36:15.129020 28389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:15.129325 28389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:15.130517 28389 hybrid_clock.cc:648] HybridClock initialized: now 1786084575130498 us; error 30 us; skew 500 ppm
Aug 07 06:36:15 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084575, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:15.133077 28389 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:36:15.133507 28389 webserver.cc:460] Webserver started at http://127.2.243.3:36809/ using document root <none> and password file <none>
I20260807 06:36:15.133728 28389 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:15.133798 28389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:15.133958 28389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:36:15.135066 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "a20d70a4fd134719ac07181a8b0fb133"
format_stamp: "Formatted at 2026-08-07 06:36:15 on dist-test-slave-rwrg"
server_key: "7c617ba9d8c73e172f9c9b179c63a55d"
server_key_iv: "7bc1d6cac6deba94e3e16afe0096eaf2"
server_key_version: "encryptionkey@0"
I20260807 06:36:15.135464 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "a20d70a4fd134719ac07181a8b0fb133"
format_stamp: "Formatted at 2026-08-07 06:36:15 on dist-test-slave-rwrg"
server_key: "7c617ba9d8c73e172f9c9b179c63a55d"
server_key_iv: "7bc1d6cac6deba94e3e16afe0096eaf2"
server_key_version: "encryptionkey@0"
I20260807 06:36:15.136991 28389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:36:15.137802 28413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:15.137969 28389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:36:15.138088 28389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "a20d70a4fd134719ac07181a8b0fb133"
format_stamp: "Formatted at 2026-08-07 06:36:15 on dist-test-slave-rwrg"
server_key: "7c617ba9d8c73e172f9c9b179c63a55d"
server_key_iv: "7bc1d6cac6deba94e3e16afe0096eaf2"
server_key_version: "encryptionkey@0"
I20260807 06:36:15.138159 28389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:15.153592 28389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:15.156118 28389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:15.156294 28389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:15.156615 28389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:15.156948 28389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:36:15.157003 28389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:15.157049 28389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:36:15.157081 28389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:15.164131 28389 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:44857
I20260807 06:36:15.164179 28579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:44857 every 8 connection(s)
I20260807 06:36:15.164634 28389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:36:15 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084575, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:36:15.173579 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.165398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:47411 (local address 127.2.243.62:39945)
0807 06:36:15.165446 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.165448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.165655 (+ 207us) server_negotiation.cc:407] Connection header received
0807 06:36:15.166143 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.166144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.166155 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.166228 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:15.167116 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.167622 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.168698 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.168894 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.168959 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.171187 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:15.171189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:15.171189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:15.171196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:15.172389 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:15.172800 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:15.172801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:15.172801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:15.172854 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:15.173065 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:15.173066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:15.173066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:15.173182 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:15.173233 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.173451 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.173514 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:36:15.173768 28582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.165359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:47411)
0807 06:36:15.165516 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.165559 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:15.166044 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:15.166280 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:15.166283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:15.166842 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:15.167028 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.167032 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.167707 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.167708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.168602 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.168604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.168969 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.168970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.169068 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.169526 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:15.169534 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:15.171099 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:15.172458 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:15.172463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:15.172465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:15.172731 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:15.172911 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:15.172911 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:15.172912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:15.173014 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:15.173234 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:15.173238 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:15.173344 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 06:36:15.173424 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:36:15.173542 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260807 06:36:15.174257 28580 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39945
I20260807 06:36:15.174361 28580 heartbeater.cc:464] Registering TS with master...
I20260807 06:36:15.174521 28580 heartbeater.cc:511] Master 127.2.243.62:39945 requested a full tablet report, sending...
I20260807 06:36:15.174721 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 28389
I20260807 06:36:15.174822 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:36:15.174949 27856 ts_manager.cc:194] Registered new tserver with Master: a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
I20260807 06:36:15.175027 3020 external_mini_cluster.cc:1428] Setting key 564b5183f2ed143d05b6b13db6498f77
I20260807 06:36:15.175405 27856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:47411
I20260807 06:36:15.176019 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:36:15.183140 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.177358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.2.243.62:39945)
0807 06:36:15.177428 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.177430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.177549 (+ 119us) server_negotiation.cc:407] Connection header received
0807 06:36:15.177570 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.177571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.177580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.177637 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:15.178144 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.178640 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.179326 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.179536 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.179597 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.180772 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:15.180773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:15.180773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:15.180780 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:15.182182 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:15.182508 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:15.182508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:15.182509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:15.182563 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:15.182704 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:15.182705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:15.182705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:15.182851 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:15.182934 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.183017 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.183081 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:36:15.184705 27856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:36:15.186323 27856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:36:15.192824 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.188729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:34349 (local address 127.0.0.1:55352)
0807 06:36:15.188831 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.188872 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:15.188941 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:15.189239 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:15.189241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:15.189244 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:15.189309 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.189311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.190370 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.190371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.191641 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.191644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.192446 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.192447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.192566 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.192583 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 06:36:15.192690 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:36:15.192746 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
I20260807 06:36:15.192838 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.188858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.2.243.2:34349)
0807 06:36:15.189044 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.189048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.189057 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:36:15.189097 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.189098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.189110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.189234 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:15.189417 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.190266 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.191736 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.192342 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.192441 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.192537 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.192706 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.192750 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260807 06:36:15.193161 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.188600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.2.243.3:44857)
0807 06:36:15.189390 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.189394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.189404 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:36:15.189435 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.189437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.189453 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.189523 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:15.189721 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.190815 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.192061 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.192768 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.192829 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.192877 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.193040 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.193081 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":131,"threads_started":1}
I20260807 06:36:15.193249 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.188631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.2.243.1:40043)
0807 06:36:15.189095 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.189099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.189107 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:36:15.189160 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.189162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.189178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.189273 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:15.189523 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.190695 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.191888 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.192618 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.192690 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.192743 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.193121 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.193174 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":96,"threads_started":1}
I20260807 06:36:15.193041 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.188537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40043 (local address 127.0.0.1:34252)
0807 06:36:15.188831 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.188872 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:15.188941 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:15.189380 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:15.189380 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:15.189382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:15.189433 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.189435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.190805 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.190806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.191814 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.191816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.192783 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.192784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.192853 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.192856 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:36:15.192912 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 06:36:15.192950 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260807 06:36:15.193136 28599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.188527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44857 (local address 127.0.0.1:40778)
0807 06:36:15.188831 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.188872 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:15.188941 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:15.189589 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:15.189590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:15.189593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:15.189635 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.189636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.190924 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.190925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.191951 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.191954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.192908 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.192909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.192959 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.192962 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:36:15.193015 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 06:36:15.193046 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":51,"threads_started":1}
I20260807 06:36:15.193966 28276 tablet_service.cc:1467] Processing CreateTablet for tablet 6abb636b63c044ce8bd255a643b664c6 (DEFAULT_TABLE table=test-table [id=63cd1ec94af04696b83a1846c0a5f4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:36:15.194110 28057 tablet_service.cc:1467] Processing CreateTablet for tablet 6abb636b63c044ce8bd255a643b664c6 (DEFAULT_TABLE table=test-table [id=63cd1ec94af04696b83a1846c0a5f4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:36:15.194365 28276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6abb636b63c044ce8bd255a643b664c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:36:15.194401 28057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6abb636b63c044ce8bd255a643b664c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:36:15.197081 28479 tablet_service.cc:1467] Processing CreateTablet for tablet 6abb636b63c044ce8bd255a643b664c6 (DEFAULT_TABLE table=test-table [id=63cd1ec94af04696b83a1846c0a5f4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:36:15.197445 28479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6abb636b63c044ce8bd255a643b664c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:36:15.197835 28603 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Bootstrap starting.
I20260807 06:36:15.198041 28604 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Bootstrap starting.
I20260807 06:36:15.198567 28603 tablet_bootstrap.cc:654] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Neither blocks nor log segments found. Creating new log.
I20260807 06:36:15.198705 28604 tablet_bootstrap.cc:654] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Neither blocks nor log segments found. Creating new log.
I20260807 06:36:15.199231 28603 log.cc:824] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:15.199301 28604 log.cc:824] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:15.199918 28604 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: No bootstrap required, opened a new log
I20260807 06:36:15.199967 28603 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: No bootstrap required, opened a new log
I20260807 06:36:15.199993 28604 ts_tablet_manager.cc:1397] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:15.200098 28603 ts_tablet_manager.cc:1397] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:15.200547 28609 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Bootstrap starting.
I20260807 06:36:15.201272 28609 tablet_bootstrap.cc:654] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Neither blocks nor log segments found. Creating new log.
I20260807 06:36:15.201822 28609 log.cc:824] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:15.201849 28603 raft_consensus.cc:348] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.202056 28603 raft_consensus.cc:374] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:36:15.202111 28603 raft_consensus.cc:729] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d5be6ce3d2436d9fb544ec0a26c2f0, State: Initialized, Role: FOLLOWER
I20260807 06:36:15.202279 28603 consensus_queue.cc:260] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.202283 28604 raft_consensus.cc:348] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.202466 28604 raft_consensus.cc:374] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:36:15.202504 28604 raft_consensus.cc:729] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a503483943f74de19ef8cb43cbb08d80, State: Initialized, Role: FOLLOWER
I20260807 06:36:15.202611 28603 ts_tablet_manager.cc:1428] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:15.202623 28604 consensus_queue.cc:260] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.202718 28609 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: No bootstrap required, opened a new log
I20260807 06:36:15.202721 28380 heartbeater.cc:503] Master 127.2.243.62:39945 was elected leader, sending a full tablet report...
I20260807 06:36:15.202792 28609 ts_tablet_manager.cc:1397] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:15.202919 28604 ts_tablet_manager.cc:1428] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:36:15.203053 28165 heartbeater.cc:503] Master 127.2.243.62:39945 was elected leader, sending a full tablet report...
I20260807 06:36:15.204730 28609 raft_consensus.cc:348] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.204849 28609 raft_consensus.cc:374] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:36:15.204869 28609 raft_consensus.cc:729] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a20d70a4fd134719ac07181a8b0fb133, State: Initialized, Role: FOLLOWER
I20260807 06:36:15.204989 28609 consensus_queue.cc:260] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.205241 28609 ts_tablet_manager.cc:1428] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:15.205276 28580 heartbeater.cc:503] Master 127.2.243.62:39945 was elected leader, sending a full tablet report...
I20260807 06:36:15.229568 28614 raft_consensus.cc:482] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:36:15.229681 28614 raft_consensus.cc:504] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.230028 28614 leader_election.cc:288] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043), a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
I20260807 06:36:15.232758 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.230216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:53881 (local address 127.2.243.1:40043)
0807 06:36:15.230261 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.230263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.230288 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:36:15.230359 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.230360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.230377 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.230459 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:15.230595 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.231042 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.232007 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.232376 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.232455 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.232465 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.232662 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.232698 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:36:15.232759 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.230166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40043 (local address 127.2.243.2:53881)
0807 06:36:15.230235 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.230247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:15.230282 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:15.230463 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:15.230464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:15.230465 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:15.230530 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.230532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.231123 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.231124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.231919 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.231922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.232464 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.232465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.232575 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.232578 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:36:15.232649 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:36:15.232689 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":37}
I20260807 06:36:15.232833 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.230270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:35083 (local address 127.2.243.3:44857)
0807 06:36:15.230310 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.230312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.230451 (+ 139us) server_negotiation.cc:407] Connection header received
0807 06:36:15.230516 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.230516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.230527 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.230589 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:15.230729 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.231161 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.232081 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.232440 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.232560 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.232569 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.232723 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.232757 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:36:15.232888 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.230190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44857 (local address 127.2.243.2:35083)
0807 06:36:15.230351 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.230362 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:15.230455 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:15.230594 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:15.230594 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:15.230596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:15.230665 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.230666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.231241 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.231242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.231989 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:15.231991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.232570 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.232572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:15.232642 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.232645 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:36:15.232717 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:36:15.232755 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":116,"threads_started":1}
I20260807 06:36:15.233088 28084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abb636b63c044ce8bd255a643b664c6" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a503483943f74de19ef8cb43cbb08d80" is_pre_election: true
I20260807 06:36:15.233182 28509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abb636b63c044ce8bd255a643b664c6" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a20d70a4fd134719ac07181a8b0fb133" is_pre_election: true
I20260807 06:36:15.233244 28084 raft_consensus.cc:2454] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 0.
I20260807 06:36:15.233300 28509 raft_consensus.cc:2454] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 0.
I20260807 06:36:15.233444 28202 leader_election.cc:302] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a503483943f74de19ef8cb43cbb08d80, b5d5be6ce3d2436d9fb544ec0a26c2f0; no voters:
I20260807 06:36:15.233615 28614 raft_consensus.cc:2781] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:36:15.233669 28614 raft_consensus.cc:482] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:36:15.233691 28614 raft_consensus.cc:3037] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:36:15.234401 28614 raft_consensus.cc:504] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.234535 28614 leader_election.cc:288] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 election: Requested vote from peers a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043), a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
I20260807 06:36:15.234687 28509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abb636b63c044ce8bd255a643b664c6" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a20d70a4fd134719ac07181a8b0fb133"
I20260807 06:36:15.234699 28084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abb636b63c044ce8bd255a643b664c6" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a503483943f74de19ef8cb43cbb08d80"
I20260807 06:36:15.234762 28084 raft_consensus.cc:3037] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:36:15.234762 28509 raft_consensus.cc:3037] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:36:15.235582 28509 raft_consensus.cc:2454] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 1.
I20260807 06:36:15.235582 28084 raft_consensus.cc:2454] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 1.
I20260807 06:36:15.235772 28204 leader_election.cc:302] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0; no voters:
I20260807 06:36:15.235864 28614 raft_consensus.cc:2781] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:36:15.236012 28614 raft_consensus.cc:686] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 LEADER]: Becoming Leader. State: Replica: b5d5be6ce3d2436d9fb544ec0a26c2f0, State: Running, Role: LEADER
I20260807 06:36:15.236102 28614 consensus_queue.cc:237] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:15.236860 27853 catalog_manager.cc:5495] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2). New cstate: current_term: 1 leader_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } health_report { overall_health: HEALTHY } } }
I20260807 06:36:15.249876 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.247740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.2.243.2:34349)
0807 06:36:15.247783 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:15.247785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:15.247793 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:36:15.247846 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:15.247846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:15.247861 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:15.247915 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:36:15.248016 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.248459 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.249249 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:15.249424 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:15.249474 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:15.249607 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:36:15.249691 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:36:15.249749 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:15.249783 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 06:36:15.249799 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:36:15.251384 28509 raft_consensus.cc:1261] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Refusing update from remote peer b5d5be6ce3d2436d9fb544ec0a26c2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:36:15.251401 28084 raft_consensus.cc:1261] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Refusing update from remote peer b5d5be6ce3d2436d9fb544ec0a26c2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:36:15.251691 28622 consensus_queue.cc:1035] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:36:15.251811 28614 consensus_queue.cc:1035] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:36:15.254392 28636 mvcc.cc:205] Tried to move back new op lower bound from 7315802420227538944 to 7315802420167622656. Current Snapshot: MvccSnapshot[applied={T|T < 7315802420227538944}]
I20260807 06:36:15.254825 28637 mvcc.cc:205] Tried to move back new op lower bound from 7315802420227538944 to 7315802420167622656. Current Snapshot: MvccSnapshot[applied={T|T < 7315802420227538944}]
I20260807 06:36:15.256201 27856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:36:15.256294 27856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:36:15.256980 27856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Sending DeleteTablet for 3 replicas of tablet 6abb636b63c044ce8bd255a643b664c6
I20260807 06:36:15.257292 28479 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abb636b63c044ce8bd255a643b664c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:36:15 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:40778
I20260807 06:36:15.257341 28057 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abb636b63c044ce8bd255a643b664c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:36:15 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:34252
I20260807 06:36:15.257371 28276 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abb636b63c044ce8bd255a643b664c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:36:15 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:55352
I20260807 06:36:15.257575 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 27957
I20260807 06:36:15.257682 28647 tablet_replica.cc:330] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: stopping tablet replica
I20260807 06:36:15.257767 28647 raft_consensus.cc:2229] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:36:15.257838 28647 raft_consensus.cc:2258] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:36:15.260371 28647 ts_tablet_manager.cc:1905] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:36:15.262027 28647 ts_tablet_manager.cc:1918] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:36:15.262105 28647 log.cc:1196] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/6abb636b63c044ce8bd255a643b664c6
I20260807 06:36:15.262353 28647 ts_tablet_manager.cc:1939] T 6abb636b63c044ce8bd255a643b664c6 P a20d70a4fd134719ac07181a8b0fb133: Deleting consensus metadata
I20260807 06:36:15.262706 27838 catalog_manager.cc:4841] TS a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857): tablet 6abb636b63c044ce8bd255a643b664c6 (table test-table [id=63cd1ec94af04696b83a1846c0a5f4dc]) successfully deleted
I20260807 06:36:15.268205 28649 tablet_replica.cc:330] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: stopping tablet replica
I20260807 06:36:15.268323 28649 raft_consensus.cc:2229] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:36:15.268443 28649 raft_consensus.cc:2258] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:36:15.269636 28649 ts_tablet_manager.cc:1905] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:36:15.271347 28649 ts_tablet_manager.cc:1918] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:36:15.271431 28649 log.cc:1196] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/6abb636b63c044ce8bd255a643b664c6
I20260807 06:36:15.271683 28649 ts_tablet_manager.cc:1939] T 6abb636b63c044ce8bd255a643b664c6 P b5d5be6ce3d2436d9fb544ec0a26c2f0: Deleting consensus metadata
I20260807 06:36:15.272190 27837 catalog_manager.cc:4841] TS b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349): tablet 6abb636b63c044ce8bd255a643b664c6 (table test-table [id=63cd1ec94af04696b83a1846c0a5f4dc]) successfully deleted
W20260807 06:36:15.275825 27837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:40043 (error 108)
I20260807 06:36:15.276244 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 28174
I20260807 06:36:15.276285 28599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:15.276161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40043 (local address 127.0.0.1:34266)
0807 06:36:15.276218 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:15.276254 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:36:15.284643 27837 catalog_manager.cc:4584] TS a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6abb636b63c044ce8bd255a643b664c6: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40043: connect: Connection refused (error 111)
I20260807 06:36:15.285643 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 28389
I20260807 06:36:15.301124 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 27807
I20260807 06:36:35.314347 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39945
--webserver_interface=127.2.243.62
--webserver_port=40913
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:36:35.399773 28652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:35.399988 28652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:35.400043 28652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:35.401247 28652 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:36:35.401311 28652 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:36:35.401335 28652 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:36:35.401353 28652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:36:35.401386 28652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:35.401410 28652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:35.401443 28652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:36:35.401473 28652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:36:35.402976 28652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=40913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:35.403265 28652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:35.403486 28652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:35.413208 28660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:35.413220 28661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:35.413234 28652 server_base.cc:1034] running on GCE node
W20260807 06:36:35.413209 28664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:35.413761 28652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:35.414126 28652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:35.415294 28652 hybrid_clock.cc:648] HybridClock initialized: now 1786084595415282 us; error 35 us; skew 500 ppm
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:35.417929 28652 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:36:35.418406 28652 webserver.cc:460] Webserver started at http://127.2.243.62:40913/ using document root <none> and password file <none>
I20260807 06:36:35.418622 28652 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:35.418676 28652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:35.420017 28652 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:35.420740 28678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.420970 28652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:36:35.421066 28652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "092291564fcc464d83fd4035f6b59070"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "64d1e821bef22195cecf5ad6a43965f2"
server_key_iv: "e814c36603662e112cfa7a5182a76ef8"
server_key_version: "encryptionkey@0"
I20260807 06:36:35.421452 28652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:35.430357 28652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:35.432286 28652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:35.432392 28652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:35.436914 28652 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39945
I20260807 06:36:35.436971 28762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39945 every 8 connection(s)
I20260807 06:36:35.437332 28652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:36:35.438833 28763 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:36:35.439520 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Bootstrap starting.
I20260807 06:36:35.440128 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 28652
I20260807 06:36:35.440440 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:40043
--local_ip_for_outbound_sockets=127.2.243.1
--tserver_master_addrs=127.2.243.62:39945
--webserver_port=45259
--webserver_interface=127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:36:35.448256 28763 log.cc:824] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:35.450454 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:36:35.450819 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Bootstrap complete.
I20260807 06:36:35.453339 28763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:35.453603 28763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 092291564fcc464d83fd4035f6b59070, State: Initialized, Role: FOLLOWER
I20260807 06:36:35.453792 28763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:35.453898 28763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:36:35.453962 28763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:36:35.454016 28763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:36:35.455081 28763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:35.455209 28763 leader_election.cc:302] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 092291564fcc464d83fd4035f6b59070; no voters:
I20260807 06:36:35.455420 28763 leader_election.cc:288] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:36:35.455461 28768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:36:35.455587 28768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 LEADER]: Becoming Leader. State: Replica: 092291564fcc464d83fd4035f6b59070, State: Running, Role: LEADER
I20260807 06:36:35.455682 28768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:35.455722 28763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:36:35.455994 28768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 092291564fcc464d83fd4035f6b59070. Latest consensus state: current_term: 2 leader_uuid: "092291564fcc464d83fd4035f6b59070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } } }
I20260807 06:36:35.456097 28768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: This master's current role is: LEADER
I20260807 06:36:35.455996 28769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "092291564fcc464d83fd4035f6b59070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } } }
I20260807 06:36:35.456305 28769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: This master's current role is: LEADER
I20260807 06:36:35.456336 28779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:36:35.457275 28779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:36:35.457444 28779 catalog_manager.cc:1259] Loaded cluster ID: 4c958c67d0aa474f9c3ef3f7a843114a
I20260807 06:36:35.457508 28779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:36:35.458745 28779 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:36:35.458956 28779 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Loaded TSK: 0
I20260807 06:36:35.468878 28779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Generated new TSK 1
I20260807 06:36:35.468999 28779 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:36:35.536695 28766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:35.536931 28766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:35.536972 28766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:35.538156 28766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:35.538224 28766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:35.538268 28766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:36:35.539707 28766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:40043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=45259
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:35.539958 28766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:35.540215 28766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:35.546330 28813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:35.546391 28816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:35.546344 28810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:35.546794 28766 server_base.cc:1034] running on GCE node
I20260807 06:36:35.546991 28766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:35.547273 28766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:35.548466 28766 hybrid_clock.cc:648] HybridClock initialized: now 1786084595548446 us; error 32 us; skew 500 ppm
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:35.551461 28766 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:36:35.552012 28766 webserver.cc:460] Webserver started at http://127.2.243.1:45259/ using document root <none> and password file <none>
I20260807 06:36:35.552289 28766 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:35.552373 28766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:35.554184 28766 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:35.555048 28826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.555269 28766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:35.555372 28766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "a503483943f74de19ef8cb43cbb08d80"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "e4bf08fa5b6708a0c1b462719c67f481"
server_key_iv: "fa41f593499ae376e7e5418f9c9bc16a"
server_key_version: "encryptionkey@0"
I20260807 06:36:35.555814 28766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:35.575758 28766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:35.578472 28766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:35.578660 28766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:35.578923 28766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:35.579532 28837 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:36:35.581364 28766 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:36:35.581429 28766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:36:35.581461 28766 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:36:35.582310 28766 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:36:35.582361 28766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:36:35.582444 28837 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Bootstrap starting.
I20260807 06:36:35.589700 28766 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40043
I20260807 06:36:35.589972 29007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40043 every 8 connection(s)
I20260807 06:36:35.590188 28766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:36:35.593477 28837 log.cc:824] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:35.595605 28837 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:36:35.595921 28837 tablet_bootstrap.cc:492] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Bootstrap complete.
I20260807 06:36:35.596809 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 28766
I20260807 06:36:35.597116 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:34349
--local_ip_for_outbound_sockets=127.2.243.2
--tserver_master_addrs=127.2.243.62:39945
--webserver_port=44509
--webserver_interface=127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:36:35.597203 28837 ts_tablet_manager.cc:1397] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260807 06:36:35.598912 28837 raft_consensus.cc:348] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:35.599255 28837 raft_consensus.cc:729] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a503483943f74de19ef8cb43cbb08d80, State: Initialized, Role: FOLLOWER
I20260807 06:36:35.599421 28837 consensus_queue.cc:260] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } }
I20260807 06:36:35.599721 28837 ts_tablet_manager.cc:1428] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:36:35.604803 29014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.591032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:58183 (local address 127.2.243.62:39945)
0807 06:36:35.592660 (+ 1628us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.592670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.592719 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:36:35.592883 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.592892 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.592956 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.593132 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:35.593943 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.595330 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.597049 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.597649 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.597758 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.600478 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:35.600482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:35.600483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:35.600516 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:35.602552 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.603120 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.603124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.603124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.603253 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.603592 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.603593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.603593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.603939 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:35.604041 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.604419 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.604607 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1568,"thread_start_us":75,"threads_started":1}
I20260807 06:36:35.604820 29013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.590871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:58183)
0807 06:36:35.591091 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:35.591139 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:35.591741 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:35.593175 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:35.593180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:35.593532 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:35.593797 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.593804 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.595472 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.595473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.596939 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.596942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.597867 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.597868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.598002 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.598588 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:35.598598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:35.600344 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:35.602641 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.602649 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.602651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.602994 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.603342 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.603343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.603343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.603505 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.604027 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:35.604034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:35.604207 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 06:36:35.604337 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 06:36:35.604562 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":124,"threads_started":1}
I20260807 06:36:35.606104 29008 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39945
I20260807 06:36:35.606221 29008 heartbeater.cc:464] Registering TS with master...
I20260807 06:36:35.606448 29008 heartbeater.cc:511] Master 127.2.243.62:39945 requested a full tablet report, sending...
I20260807 06:36:35.607326 28701 ts_manager.cc:194] Registered new tserver with Master: a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043)
I20260807 06:36:35.608695 28701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:58183
I20260807 06:36:35.609815 29008 heartbeater.cc:503] Master 127.2.243.62:39945 was elected leader, sending a full tablet report...
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.1@KRBTEST.COM
I20260807 06:36:35.618208 29026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.608011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40043 (local address 127.0.0.1:56544)
0807 06:36:35.608608 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:35.608644 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:35.608708 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:35.609040 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:35.609046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:35.609382 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:35.609435 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.609438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.610942 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.610944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.612028 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.612032 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.612752 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.612754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.612832 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.613361 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:35.613366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:35.614969 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:35.617063 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.617069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.617071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.617271 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.617506 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.617507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.617507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.617576 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.617898 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:35.617905 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:35.617953 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:36:35.618035 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 06:36:35.618140 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":100,"threads_started":1}
I20260807 06:36:35.618330 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.608257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56544 (local address 127.2.243.1:40043)
0807 06:36:35.608892 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.608896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.608905 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:36:35.608933 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.608935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.608947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.609517 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:35.609680 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.610833 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.612118 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.612638 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.612706 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.615063 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:35.615071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:35.615071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:35.615079 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:35.616977 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.617352 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.617355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.617355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.617418 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.617648 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.617649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.617649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.617800 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:35.617890 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.618102 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.618197 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":64,"threads_started":1}
I20260807 06:36:35.618673 28902 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abb636b63c044ce8bd255a643b664c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:36:15 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:56544
I20260807 06:36:35.619274 29028 tablet_replica.cc:330] stopping tablet replica
I20260807 06:36:35.619397 29028 raft_consensus.cc:2229] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:36:35.619477 29028 raft_consensus.cc:2258] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:36:35.619771 29028 ts_tablet_manager.cc:1905] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:36:35.621923 29028 ts_tablet_manager.cc:1918] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:36:35.622006 29028 log.cc:1196] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal/wals/6abb636b63c044ce8bd255a643b664c6
I20260807 06:36:35.622229 29028 ts_tablet_manager.cc:1939] T 6abb636b63c044ce8bd255a643b664c6 P a503483943f74de19ef8cb43cbb08d80: Deleting consensus metadata
I20260807 06:36:35.622771 28681 catalog_manager.cc:4841] TS a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043): tablet 6abb636b63c044ce8bd255a643b664c6 (table test-table [id=63cd1ec94af04696b83a1846c0a5f4dc]) successfully deleted
W20260807 06:36:35.687114 29019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:35.687299 29019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:35.687320 29019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:35.688462 29019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:35.688526 29019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:35.688555 29019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:36:35.690006 29019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:34349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=44509
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:35.690248 29019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:35.690478 29019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:35.695472 29038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:35.695483 29039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:35.695482 29042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:35.695791 29019 server_base.cc:1034] running on GCE node
I20260807 06:36:35.695976 29019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:35.696171 29019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:35.697330 29019 hybrid_clock.cc:648] HybridClock initialized: now 1786084595697317 us; error 26 us; skew 500 ppm
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:35.699702 29019 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:36:35.700126 29019 webserver.cc:460] Webserver started at http://127.2.243.2:44509/ using document root <none> and password file <none>
I20260807 06:36:35.700371 29019 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:35.700429 29019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:35.701867 29019 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:35.702479 29053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.702636 29019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:36:35.702724 29019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "ee05bb2937cea15b8786ee591422dd08"
server_key_iv: "c3dae8318c35c3b13f92ad9a3dd07290"
server_key_version: "encryptionkey@0"
I20260807 06:36:35.703004 29019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:35.715996 29019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:35.717943 29019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:35.718062 29019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:35.718231 29019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:35.718473 29019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:36:35.718501 29019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.718523 29019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:36:35.718540 29019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.724303 29019 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:34349
I20260807 06:36:35.724391 29235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:34349 every 8 connection(s)
I20260807 06:36:35.724794 29019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:36:35.733734 29014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.725558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:44537 (local address 127.2.243.62:39945)
0807 06:36:35.725601 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.725604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.725789 (+ 185us) server_negotiation.cc:407] Connection header received
0807 06:36:35.726248 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.726249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.726258 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.726334 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:35.727225 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.727736 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.728824 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.729028 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.729090 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.731116 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:35.731118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:35.731118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:35.731125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:35.732425 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.732877 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.732879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.732879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.732939 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.733191 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.733192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.733192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.733297 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:35.733351 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.733581 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.733667 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:36:35.733879 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.725501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:44537)
0807 06:36:35.725665 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:35.725705 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:35.726170 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:35.726400 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:35.726403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:35.726924 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:35.727116 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.727120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.727829 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.727830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.728747 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.728750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.729102 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.729103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.729201 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.729687 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:35.729695 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:35.731008 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:35.732557 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.732565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.732567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.732780 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.733026 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.733027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.733027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.733129 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.733356 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:35.733361 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:35.733469 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 06:36:35.733558 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:36:35.733674 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260807 06:36:35.734481 29236 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39945
I20260807 06:36:35.734546 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 29019
I20260807 06:36:35.734652 29236 heartbeater.cc:464] Registering TS with master...
I20260807 06:36:35.734860 29236 heartbeater.cc:511] Master 127.2.243.62:39945 requested a full tablet report, sending...
I20260807 06:36:35.734869 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:44857
--local_ip_for_outbound_sockets=127.2.243.3
--tserver_master_addrs=127.2.243.62:39945
--webserver_port=36809
--webserver_interface=127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:36:35.735258 28701 ts_manager.cc:194] Registered new tserver with Master: b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349)
I20260807 06:36:35.735648 28701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:44537
W20260807 06:36:35.818235 29239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:35.818428 29239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:35.818454 29239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:35.819520 29239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:35.819566 29239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:35.819590 29239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:36:35.821125 29239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:44857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=36809
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:35.821307 29239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:35.821483 29239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:35.826777 29247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:35.826792 29246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:35.826996 29251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:35.827404 29239 server_base.cc:1034] running on GCE node
I20260807 06:36:35.827629 29239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:35.827885 29239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:35.829052 29239 hybrid_clock.cc:648] HybridClock initialized: now 1786084595829025 us; error 42 us; skew 500 ppm
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:35.831653 29239 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:36:35.832099 29239 webserver.cc:460] Webserver started at http://127.2.243.3:36809/ using document root <none> and password file <none>
I20260807 06:36:35.832396 29239 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:35.832528 29239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:35.833890 29239 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:35.834522 29259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.834705 29239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:36:35.834813 29239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "a20d70a4fd134719ac07181a8b0fb133"
format_stamp: "Formatted at 2026-08-07 06:36:15 on dist-test-slave-rwrg"
server_key: "7c617ba9d8c73e172f9c9b179c63a55d"
server_key_iv: "7bc1d6cac6deba94e3e16afe0096eaf2"
server_key_version: "encryptionkey@0"
I20260807 06:36:35.835170 29239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:35.852139 29239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:35.854557 29239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:35.854723 29239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:35.854964 29239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:35.855283 29239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:36:35.855340 29239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.855381 29239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:36:35.855410 29239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:35.861785 29239 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:44857
I20260807 06:36:35.861829 29434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:44857 every 8 connection(s)
I20260807 06:36:35.862265 29239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
Aug 07 06:36:35 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084595, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:36:35.870934 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 29239
I20260807 06:36:35.871896 29014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.863040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:42739 (local address 127.2.243.62:39945)
0807 06:36:35.863105 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.863107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.863328 (+ 221us) server_negotiation.cc:407] Connection header received
0807 06:36:35.863879 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.863880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.863892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.863957 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:35.864995 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.865518 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.866537 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.866730 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.866824 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.868933 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:35.868935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:35.868935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:35.868942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:35.870295 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.870724 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.870725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.870725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.870777 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.871082 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.871083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.871083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.871224 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:35.871298 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.871662 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.871782 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:36:35.872018 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.863009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:42739)
0807 06:36:35.863202 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:35.863238 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:35.863762 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:35.864035 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:35.864038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:35.864700 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:35.864888 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.864893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.865620 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.865621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.866430 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.866432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.866848 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.866849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.866967 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.867427 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:35.867435 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:35.868842 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:35.870371 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.870377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.870379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.870640 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.870844 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:35.870845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:35.870845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:35.870990 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:35.871317 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:35.871323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:35.871491 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 06:36:35.871629 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 06:36:35.871805 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260807 06:36:35.872540 29435 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39945
I20260807 06:36:35.872642 29435 heartbeater.cc:464] Registering TS with master...
I20260807 06:36:35.872824 29435 heartbeater.cc:511] Master 127.2.243.62:39945 requested a full tablet report, sending...
I20260807 06:36:35.873275 28701 ts_manager.cc:194] Registered new tserver with Master: a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
I20260807 06:36:35.873627 28701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:42739
I20260807 06:36:35.878134 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.871370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.2.243.62:39945)
0807 06:36:35.871504 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.871506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.871610 (+ 104us) server_negotiation.cc:407] Connection header received
0807 06:36:35.871683 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.871684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.871696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.871763 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:35.872548 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.873213 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.874185 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.874468 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.874578 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.875622 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:35.875623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:35.875623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:35.875629 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:35.877016 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.877339 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.877340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.877340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.877397 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:35.877698 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:35.877698 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:35.877699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:35.877806 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:35.877928 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.878005 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.878071 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260807 06:36:35.878968 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:36:35.881644 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.879326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46658 (local address 127.2.243.62:39945)
0807 06:36:35.879367 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.879369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.879448 (+ 79us) server_negotiation.cc:407] Connection header received
0807 06:36:35.879514 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.879515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.879543 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.879608 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:36:35.879711 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.880152 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.880975 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.881194 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.881268 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.881389 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:36:35.881479 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:36:35.881545 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.881578 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.881596 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:36:35.882025 28701 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:36:35.883548 28701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:36:35.885854 28902 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8e7a8111f04093be7b8f77c74e4c1c (DEFAULT_TABLE table=test-table [id=fd43ce6c645f4c2d853ffa18a723aae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:36:35.886086 28902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8e7a8111f04093be7b8f77c74e4c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:36:35.887678 28837 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Bootstrap starting.
I20260807 06:36:35.888106 28837 tablet_bootstrap.cc:654] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Neither blocks nor log segments found. Creating new log.
I20260807 06:36:35.888798 28837 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: No bootstrap required, opened a new log
I20260807 06:36:35.888883 28837 ts_tablet_manager.cc:1397] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 06:36:35.889257 28837 raft_consensus.cc:348] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:35.889346 28837 raft_consensus.cc:374] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:36:35.889387 28837 raft_consensus.cc:729] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a503483943f74de19ef8cb43cbb08d80, State: Initialized, Role: FOLLOWER
I20260807 06:36:35.889446 28837 consensus_queue.cc:260] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:35.889575 28837 ts_tablet_manager.cc:1428] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:36:35.889695 29026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.885527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:34349 (local address 127.0.0.1:55154)
0807 06:36:35.885666 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:35.885676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:35.885713 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:35.886106 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:35.886107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:35.886109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:35.886149 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.886150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.887311 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.887312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.888573 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.888577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.889448 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.889449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.889522 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.889547 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 06:36:35.889579 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 06:36:35.889623 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 06:36:35.890159 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.885579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55154 (local address 127.2.243.2:34349)
0807 06:36:35.885866 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.885871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.885883 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:36:35.885930 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.885933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.885951 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.886092 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:35.886258 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.887215 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.888653 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.889322 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.889893 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.889976 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.890006 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.890059 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":98,"threads_started":1}
I20260807 06:36:35.890772 29122 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8e7a8111f04093be7b8f77c74e4c1c (DEFAULT_TABLE table=test-table [id=fd43ce6c645f4c2d853ffa18a723aae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:36:35.891112 29122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8e7a8111f04093be7b8f77c74e4c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:36:35.891585 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.885582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.2.243.3:44857)
0807 06:36:35.887704 (+ 2122us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:35.887708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:35.887719 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:36:35.887753 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:35.887755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:35.887769 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:35.887849 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:35.888100 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.889434 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.890707 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.891254 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.891328 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.891366 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:35.891496 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 06:36:35.891524 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2089,"thread_start_us":103,"threads_started":1}
I20260807 06:36:35.891599 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:35.885529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44857 (local address 127.0.0.1:50912)
0807 06:36:35.885891 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:35.885902 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:35.885950 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:35.887941 (+ 1991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:35.887942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:35.887944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:35.888008 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.888009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.889569 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.889570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.890612 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:35.890615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:35.891346 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:35.891348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:35.891410 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:35.891413 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:36:35.891483 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:36:35.891525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":194,"thread_start_us":90,"threads_started":1}
I20260807 06:36:35.893242 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8e7a8111f04093be7b8f77c74e4c1c (DEFAULT_TABLE table=test-table [id=fd43ce6c645f4c2d853ffa18a723aae0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:36:35.893594 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8e7a8111f04093be7b8f77c74e4c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:36:35.894315 29450 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Bootstrap starting.
I20260807 06:36:35.895015 29450 tablet_bootstrap.cc:654] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Neither blocks nor log segments found. Creating new log.
I20260807 06:36:35.895651 29450 log.cc:824] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:35.896425 29450 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: No bootstrap required, opened a new log
I20260807 06:36:35.896531 29450 ts_tablet_manager.cc:1397] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:35.897179 29453 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Bootstrap starting.
I20260807 06:36:35.897774 29453 tablet_bootstrap.cc:654] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Neither blocks nor log segments found. Creating new log.
I20260807 06:36:35.898252 29453 log.cc:824] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:35.898167 29450 raft_consensus.cc:348] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:35.898317 29450 raft_consensus.cc:374] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:36:35.898376 29450 raft_consensus.cc:729] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d5be6ce3d2436d9fb544ec0a26c2f0, State: Initialized, Role: FOLLOWER
I20260807 06:36:35.898502 29450 consensus_queue.cc:260] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:35.898818 29450 ts_tablet_manager.cc:1428] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:35.899111 29453 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: No bootstrap required, opened a new log
I20260807 06:36:35.899191 29453 ts_tablet_manager.cc:1397] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:36:35.900631 29236 heartbeater.cc:503] Master 127.2.243.62:39945 was elected leader, sending a full tablet report...
I20260807 06:36:35.900547 29453 raft_consensus.cc:348] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:35.900734 29453 raft_consensus.cc:374] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:36:35.900779 29453 raft_consensus.cc:729] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a20d70a4fd134719ac07181a8b0fb133, State: Initialized, Role: FOLLOWER
I20260807 06:36:35.900895 29453 consensus_queue.cc:260] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:35.901196 29453 ts_tablet_manager.cc:1428] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:36:35.901192 29435 heartbeater.cc:503] Master 127.2.243.62:39945 was elected leader, sending a full tablet report...
W20260807 06:36:35.975396 29237 tablet.cc:2369] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:36:36.022102 29455 raft_consensus.cc:482] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:36:36.022228 29455 raft_consensus.cc:504] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:36.022663 29455 leader_election.cc:288] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857), a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043)
I20260807 06:36:36.026032 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.022896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44857 (local address 127.2.243.2:48265)
0807 06:36:36.022956 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:36.022971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:36.023036 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:36.023188 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:36.023190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:36.023192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:36.023266 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:36.023269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.023913 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.023914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:36.025083 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:36.025086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.025747 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.025748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:36.025831 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:36.025836 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:36:36.025906 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:36:36.025954 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 06:36:36.026069 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.022957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:48265 (local address 127.2.243.3:44857)
0807 06:36:36.023043 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:36.023046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:36.023060 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:36:36.023112 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:36.023113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:36.023126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:36.023183 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:36.023342 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.023835 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.025206 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.025665 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.025722 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:36.025731 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:36.025950 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 06:36:36.025990 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:36:36.026221 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.022920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40043 (local address 127.2.243.2:51161)
0807 06:36:36.023163 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:36.023176 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:36.023228 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:36.023424 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:36.023425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:36.023427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:36:36.023538 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:36.023539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.024381 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.024382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:36.025230 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:36.025233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.025966 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.025967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:36.026036 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:36.026040 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:36:36.026107 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:36:36.026150 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":125,"threads_started":1}
I20260807 06:36:36.026405 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.023209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:51161 (local address 127.2.243.1:40043)
0807 06:36:36.023262 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:36.023265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:36.023273 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:36:36.023298 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:36.023299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:36.023329 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:36.023420 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:36:36.023619 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.024286 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.025314 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.025872 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.025948 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:36.026040 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:36.026278 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 06:36:36.026332 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:36:36.026602 29367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8e7a8111f04093be7b8f77c74e4c1c" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a20d70a4fd134719ac07181a8b0fb133" is_pre_election: true
I20260807 06:36:36.026674 28935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8e7a8111f04093be7b8f77c74e4c1c" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a503483943f74de19ef8cb43cbb08d80" is_pre_election: true
I20260807 06:36:36.026789 29367 raft_consensus.cc:2454] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 0.
I20260807 06:36:36.026845 28935 raft_consensus.cc:2454] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 0.
I20260807 06:36:36.027052 29064 leader_election.cc:302] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0; no voters:
I20260807 06:36:36.027207 29455 raft_consensus.cc:2781] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:36:36.027303 29455 raft_consensus.cc:482] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:36:36.027345 29455 raft_consensus.cc:3037] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:36:36.028183 29455 raft_consensus.cc:504] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:36.028359 29455 leader_election.cc:288] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 election: Requested vote from peers a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857), a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043)
I20260807 06:36:36.028563 28935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8e7a8111f04093be7b8f77c74e4c1c" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a503483943f74de19ef8cb43cbb08d80"
I20260807 06:36:36.028563 29367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8e7a8111f04093be7b8f77c74e4c1c" candidate_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a20d70a4fd134719ac07181a8b0fb133"
I20260807 06:36:36.028648 28935 raft_consensus.cc:3037] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:36:36.028656 29367 raft_consensus.cc:3037] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:36:36.029421 29367 raft_consensus.cc:2454] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 1.
I20260807 06:36:36.029428 28935 raft_consensus.cc:2454] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5d5be6ce3d2436d9fb544ec0a26c2f0 in term 1.
I20260807 06:36:36.029618 29064 leader_election.cc:302] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0; no voters:
I20260807 06:36:36.029708 29455 raft_consensus.cc:2781] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:36:36.029897 29455 raft_consensus.cc:686] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 LEADER]: Becoming Leader. State: Replica: b5d5be6ce3d2436d9fb544ec0a26c2f0, State: Running, Role: LEADER
I20260807 06:36:36.029999 29455 consensus_queue.cc:237] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:36.030889 28700 catalog_manager.cc:5495] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2). New cstate: current_term: 1 leader_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } health_report { overall_health: UNKNOWN } } }
I20260807 06:36:36.071848 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.069416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.2.243.2:34349)
0807 06:36:36.069494 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:36.069497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:36.069505 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:36:36.069549 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:36.069550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:36.069571 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:36.069652 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:36:36.069766 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.070246 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.071005 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.071210 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.071301 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:36.071465 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:36:36.071517 (+ 52us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 06:36:36.071696 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55158: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 06:36:36.071923 29442 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55158: token expired
I20260807 06:36:36.078522 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.072370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.2.243.62:39945)
0807 06:36:36.072437 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:36.072439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:36.072449 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:36:36.072473 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:36.072474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:36.072483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:36.072568 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:36.073118 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.073589 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.074290 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.074477 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.074541 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:36.075790 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:36.075791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:36.075792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:36.075797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:36.077519 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:36.077875 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:36.077876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:36.077876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:36.077927 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:36.078115 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:36.078115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:36.078115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:36.078211 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:36.078286 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:36.078395 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 06:36:36.078457 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:36:36.081930 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.079490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.2.243.2:34349)
0807 06:36:36.079561 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:36.079563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:36.079572 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:36:36.079620 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:36.079621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:36.079635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:36.079714 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:36:36.079838 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.080309 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.081122 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:36.081315 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:36.081407 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:36.081571 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:36:36.081675 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:36:36.081746 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:36.081815 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 06:36:36.081837 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:36:36.083994 28935 raft_consensus.cc:1261] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Refusing update from remote peer b5d5be6ce3d2436d9fb544ec0a26c2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:36:36.084003 29367 raft_consensus.cc:1261] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Refusing update from remote peer b5d5be6ce3d2436d9fb544ec0a26c2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:36:36.084368 29455 consensus_queue.cc:1035] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:36:36.084611 29461 consensus_queue.cc:1035] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:36:36.087045 29470 mvcc.cc:205] Tried to move back new op lower bound from 7315802505556828160 to 7315802505339482112. Current Snapshot: MvccSnapshot[applied={T|T < 7315802505556828160}]
I20260807 06:36:36.088203 29474 mvcc.cc:205] Tried to move back new op lower bound from 7315802505556828160 to 7315802505339482112. Current Snapshot: MvccSnapshot[applied={T|T < 7315802505556828160}]
I20260807 06:36:36.089829 28700 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:36:36.089927 28700 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:36:36.090726 28700 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Sending DeleteTablet for 3 replicas of tablet 0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:36:36.091153 28902 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8e7a8111f04093be7b8f77c74e4c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:36:36 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:56544
I20260807 06:36:36.091178 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8e7a8111f04093be7b8f77c74e4c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:36:36 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:50912
I20260807 06:36:36.091275 29028 tablet_replica.cc:330] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: stopping tablet replica
I20260807 06:36:36.091343 29028 raft_consensus.cc:2229] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:36:36.091394 29028 raft_consensus.cc:2258] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:36:36.091502 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 28766
I20260807 06:36:36.091555 29122 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8e7a8111f04093be7b8f77c74e4c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:36:36 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:55154
I20260807 06:36:36.100150 29482 tablet_replica.cc:330] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: stopping tablet replica
I20260807 06:36:36.100318 29482 raft_consensus.cc:2229] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:36:36.100442 29482 raft_consensus.cc:2258] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:36:36.101203 29482 ts_tablet_manager.cc:1905] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:36:36.103307 29482 ts_tablet_manager.cc:1918] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:36:36.103423 29482 log.cc:1196] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:36:36.103749 29482 ts_tablet_manager.cc:1939] T 0f8e7a8111f04093be7b8f77c74e4c1c P b5d5be6ce3d2436d9fb544ec0a26c2f0: Deleting consensus metadata
I20260807 06:36:36.104350 28681 catalog_manager.cc:4841] TS b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349): tablet 0f8e7a8111f04093be7b8f77c74e4c1c (table test-table [id=fd43ce6c645f4c2d853ffa18a723aae0]) successfully deleted
I20260807 06:36:36.109054 29481 tablet_replica.cc:330] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: stopping tablet replica
I20260807 06:36:36.109194 29481 raft_consensus.cc:2229] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:36:36.109334 29481 raft_consensus.cc:2258] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:36:36.109871 29481 ts_tablet_manager.cc:1905] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:36:36.111899 29481 ts_tablet_manager.cc:1918] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:36:36.112000 29481 log.cc:1196] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:36:36.112322 29481 ts_tablet_manager.cc:1939] T 0f8e7a8111f04093be7b8f77c74e4c1c P a20d70a4fd134719ac07181a8b0fb133: Deleting consensus metadata
I20260807 06:36:36.112902 28682 catalog_manager.cc:4841] TS a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857): tablet 0f8e7a8111f04093be7b8f77c74e4c1c (table test-table [id=fd43ce6c645f4c2d853ffa18a723aae0]) successfully deleted
W20260807 06:36:36.113989 28681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:40043 (error 108)
I20260807 06:36:36.114425 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 29019
I20260807 06:36:36.114521 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:36.114366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40043 (local address 127.0.0.1:56550)
0807 06:36:36.114420 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:36.114469 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 06:36:36.118341 28681 catalog_manager.cc:4584] TS a503483943f74de19ef8cb43cbb08d80 (127.2.243.1:40043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f8e7a8111f04093be7b8f77c74e4c1c: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40043: connect: Connection refused (error 111)
I20260807 06:36:36.126835 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 29239
I20260807 06:36:36.139672 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 28652
I20260807 06:36:56.154060 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39945
--webserver_interface=127.2.243.62
--webserver_port=40913
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:36:56.237720 29485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:56.237922 29485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:56.237944 29485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:56.239357 29485 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:36:56.239418 29485 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:36:56.239439 29485 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:36:56.239470 29485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:36:56.239487 29485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:56.239521 29485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:56.239552 29485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:36:56.239588 29485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:36:56.242141 29485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=40913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:56.242425 29485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:56.242718 29485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:56.249706 29493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:56.249706 29495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:56.249729 29485 server_base.cc:1034] running on GCE node
W20260807 06:36:56.250000 29492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:56.250270 29485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:56.250617 29485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:56.251796 29485 hybrid_clock.cc:648] HybridClock initialized: now 1786084616251784 us; error 29 us; skew 500 ppm
Aug 07 06:36:56 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084616, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:56.254285 29485 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:36:56.254701 29485 webserver.cc:460] Webserver started at http://127.2.243.62:40913/ using document root <none> and password file <none>
I20260807 06:36:56.254933 29485 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:56.255003 29485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:56.256340 29485 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:56.257051 29505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.257218 29485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.257321 29485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "092291564fcc464d83fd4035f6b59070"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "64d1e821bef22195cecf5ad6a43965f2"
server_key_iv: "e814c36603662e112cfa7a5182a76ef8"
server_key_version: "encryptionkey@0"
I20260807 06:36:56.257671 29485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:56.268361 29485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:56.270359 29485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:56.270531 29485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:56.274528 29485 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39945
I20260807 06:36:56.274591 29588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39945 every 8 connection(s)
I20260807 06:36:56.274979 29485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:36:56.276696 29589 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:36:56.277408 29589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Bootstrap starting.
I20260807 06:36:56.280045 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 29485
I20260807 06:36:56.280525 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:40043
--local_ip_for_outbound_sockets=127.2.243.1
--tserver_master_addrs=127.2.243.62:39945
--webserver_port=45259
--webserver_interface=127.2.243.1
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:36:56.286518 29589 log.cc:824] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:56.289088 29589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:36:56.289449 29589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Bootstrap complete.
I20260807 06:36:56.291702 29589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:56.291934 29589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 092291564fcc464d83fd4035f6b59070, State: Initialized, Role: FOLLOWER
I20260807 06:36:56.292115 29589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:56.292186 29589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:36:56.292238 29589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:36:56.292292 29589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:36:56.293385 29589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:56.293505 29589 leader_election.cc:302] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 092291564fcc464d83fd4035f6b59070; no voters:
I20260807 06:36:56.293689 29589 leader_election.cc:288] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 06:36:56.293733 29595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:36:56.293870 29595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [term 3 LEADER]: Becoming Leader. State: Replica: 092291564fcc464d83fd4035f6b59070, State: Running, Role: LEADER
I20260807 06:36:56.293969 29589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:36:56.293963 29595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } }
I20260807 06:36:56.294320 29595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 092291564fcc464d83fd4035f6b59070. Latest consensus state: current_term: 3 leader_uuid: "092291564fcc464d83fd4035f6b59070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } } }
I20260807 06:36:56.294425 29595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: This master's current role is: LEADER
I20260807 06:36:56.294322 29596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "092291564fcc464d83fd4035f6b59070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092291564fcc464d83fd4035f6b59070" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39945 } } }
I20260807 06:36:56.294926 29596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070 [sys.catalog]: This master's current role is: LEADER
I20260807 06:36:56.294951 29604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:36:56.295938 29604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:36:56.296074 29604 catalog_manager.cc:1259] Loaded cluster ID: 4c958c67d0aa474f9c3ef3f7a843114a
I20260807 06:36:56.296159 29604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:36:56.297247 29604 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:36:56.297458 29604 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Loaded TSK: 0
I20260807 06:36:56.297533 29604 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 092291564fcc464d83fd4035f6b59070: Loaded TSK: 1
F20260807 06:36:56.297765 29604 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f819d6db9ad google::LogMessage::Fail() at ??:0
@ 0x7f819d6de88a google::LogMessage::SendToLog() at ??:0
@ 0x7f819d6db4de google::LogMessage::Flush() at ??:0
@ 0x7f819d6dca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f819fe48104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f819fe45d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f81a56351cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f81a563548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f81a5635594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f81a55fbc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f81a5605ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f819f5a90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f819f59e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f81a04026db start_thread at ??:0
@ 0x7f819c1d961f clone at ??:0
W20260807 06:36:56.386765 29592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:56.386974 29592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:56.386999 29592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:56.388182 29592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:56.388228 29592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:56.388257 29592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:36:56.389843 29592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:40043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=45259
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:56.390094 29592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:56.390344 29592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:56.395606 29656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:56.395474 29658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:56.395733 29592 server_base.cc:1034] running on GCE node
W20260807 06:36:56.395474 29655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:56.396077 29592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:56.396335 29592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:56.397544 29592 hybrid_clock.cc:648] HybridClock initialized: now 1786084616397524 us; error 37 us; skew 500 ppm
Aug 07 06:36:56 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084616, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:56.400307 29592 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:36:56.400882 29592 webserver.cc:460] Webserver started at http://127.2.243.1:45259/ using document root <none> and password file <none>
I20260807 06:36:56.401140 29592 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:56.401211 29592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:56.402685 29592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:36:56.403406 29671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.403600 29592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:56.403684 29592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "a503483943f74de19ef8cb43cbb08d80"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "e4bf08fa5b6708a0c1b462719c67f481"
server_key_iv: "fa41f593499ae376e7e5418f9c9bc16a"
server_key_version: "encryptionkey@0"
I20260807 06:36:56.404039 29592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:56.424109 29592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:56.426362 29592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:56.426514 29592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:56.426795 29592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:56.427304 29678 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:36:56.428907 29592 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:36:56.428963 29592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 06:36:56.428989 29592 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:36:56.429733 29592 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:36:56.429780 29592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 06:36:56.429823 29678 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Bootstrap starting.
I20260807 06:36:56.437561 29592 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40043
I20260807 06:36:56.437719 29855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40043 every 8 connection(s)
I20260807 06:36:56.438104 29592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:36:56.440971 29678 log.cc:824] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Log is configured to *not* fsync() on all Append() calls
I20260807 06:36:56.443131 29678 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:36:56.443441 29678 tablet_bootstrap.cc:492] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Bootstrap complete.
I20260807 06:36:56.444329 29678 ts_tablet_manager.cc:1397] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260807 06:36:56.445391 29678 raft_consensus.cc:348] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:56.445611 29678 raft_consensus.cc:729] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a503483943f74de19ef8cb43cbb08d80, State: Initialized, Role: FOLLOWER
I20260807 06:36:56.445746 29678 consensus_queue.cc:260] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:56.446025 29678 ts_tablet_manager.cc:1428] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260807 06:36:56.447301 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 29592
I20260807 06:36:56.447614 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:34349
--local_ip_for_outbound_sockets=127.2.243.2
--tserver_master_addrs=127.2.243.62:39945
--webserver_port=44509
--webserver_interface=127.2.243.2
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:36:56.491788 29860 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:36:56.438770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:51975)
0807 06:36:56.439030 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.439068 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:56.439689 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:56.491551 (+ 51862us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":92,"threads_started":1}
W20260807 06:36:56.492038 29857 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:39945 (0 consecutive failures): Network error: Failed to ping master at 127.2.243.62:39945: Client connection negotiation failed: client connection to 127.2.243.62:39945: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:36:56.492383 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.492254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:33839)
0807 06:36:56.492309 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.492353 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:36:56.492707 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.492617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:35289)
0807 06:36:56.492662 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.492686 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 06:36:56.492820 29857 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:36:56.533418 29866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:56.533668 29866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:56.533710 29866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:56.534807 29866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:56.534879 29866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:56.534927 29866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:36:56.536406 29866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:34349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=44509
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:56.536710 29866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:56.537081 29866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:56.542289 29873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:56.542304 29878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:56.542315 29874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:56.542382 29866 server_base.cc:1034] running on GCE node
I20260807 06:36:56.542706 29866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:56.542951 29866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:56.544118 29866 hybrid_clock.cc:648] HybridClock initialized: now 1786084616544120 us; error 45 us; skew 500 ppm
Aug 07 06:36:56 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084616, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:56.546746 29866 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:36:56.547169 29866 webserver.cc:460] Webserver started at http://127.2.243.2:44509/ using document root <none> and password file <none>
I20260807 06:36:56.547398 29866 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:56.547467 29866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:56.548888 29866 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:56.549556 29892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.549744 29866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:56.549856 29866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0"
format_stamp: "Formatted at 2026-08-07 06:36:14 on dist-test-slave-rwrg"
server_key: "ee05bb2937cea15b8786ee591422dd08"
server_key_iv: "c3dae8318c35c3b13f92ad9a3dd07290"
server_key_version: "encryptionkey@0"
I20260807 06:36:56.550197 29866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:56.563236 29866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:56.565172 29866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:56.565342 29866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:56.565604 29866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:56.565997 29866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:36:56.566056 29866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.566092 29866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:36:56.566118 29866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.572125 29866 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:34349
I20260807 06:36:56.572180 30078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:34349 every 8 connection(s)
I20260807 06:36:56.572546 29866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:36:56.573247 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 29866
I20260807 06:36:56.573551 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:44857
--local_ip_for_outbound_sockets=127.2.243.3
--tserver_master_addrs=127.2.243.62:39945
--webserver_port=36809
--webserver_interface=127.2.243.3
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:36:56.573644 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.573202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:51589)
0807 06:36:56.573371 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.573498 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
W20260807 06:36:56.574007 30079 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:39945 (0 consecutive failures): Network error: Failed to ping master at 127.2.243.62:39945: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
I20260807 06:36:56.574282 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.574184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:36549)
0807 06:36:56.574233 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.574257 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 06:36:56.574611 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.574482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:40935)
0807 06:36:56.574575 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.574596 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 06:36:56.574714 30079 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:36:56.659682 30086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:36:56.659893 30086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:36:56.659916 30086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:36:56.661096 30086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:36:56.661139 30086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:36:56.661165 30086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:36:56.662668 30086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:44857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=36809
--tserver_master_addrs=127.2.243.62:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:36:56.662912 30086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:36:56.663156 30086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:36:56.668358 30095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:56.668354 30094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:36:56.668355 30097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:36:56.668846 30086 server_base.cc:1034] running on GCE node
I20260807 06:36:56.669078 30086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:36:56.669335 30086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:36:56.670531 30086 hybrid_clock.cc:648] HybridClock initialized: now 1786084616670522 us; error 37 us; skew 500 ppm
Aug 07 06:36:56 dist-test-slave-rwrg krb5kdc[27784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084616, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:36:56.673219 30086 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:36:56.673686 30086 webserver.cc:460] Webserver started at http://127.2.243.3:36809/ using document root <none> and password file <none>
I20260807 06:36:56.673985 30086 fs_manager.cc:359] Metadata directory not provided
I20260807 06:36:56.674041 30086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:36:56.675381 30086 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:56.676018 30111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.676208 30086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:36:56.676301 30086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "a20d70a4fd134719ac07181a8b0fb133"
format_stamp: "Formatted at 2026-08-07 06:36:15 on dist-test-slave-rwrg"
server_key: "7c617ba9d8c73e172f9c9b179c63a55d"
server_key_iv: "7bc1d6cac6deba94e3e16afe0096eaf2"
server_key_version: "encryptionkey@0"
I20260807 06:36:56.676702 30086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:36:56.698261 30086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:36:56.700137 30086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:36:56.700245 30086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:36:56.700421 30086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:36:56.700805 30086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:36:56.700839 30086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.700886 30086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:36:56.700914 30086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:36:56.706811 30086 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:44857
I20260807 06:36:56.706890 30305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:44857 every 8 connection(s)
I20260807 06:36:56.707237 30086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:36:56.708405 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.707963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:34221)
0807 06:36:56.708179 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.708265 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":136,"threads_started":1}
W20260807 06:36:56.708835 30307 heartbeater.cc:644] Failed to heartbeat to 127.2.243.62:39945 (0 consecutive failures): Network error: Failed to ping master at 127.2.243.62:39945: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
I20260807 06:36:56.709131 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.709041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:37761)
0807 06:36:56.709086 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.709112 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 06:36:56.709381 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:56.709333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:35963)
0807 06:36:56.709352 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:56.709369 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260807 06:36:56.709513 30307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:36:56.709913 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 30086
I20260807 06:36:57.493407 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:57.493156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:59293)
0807 06:36:57.493298 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:57.493336 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260807 06:36:57.575271 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:57.575050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:59161)
0807 06:36:57.575191 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:57.575226 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260807 06:36:57.602095 30315 raft_consensus.cc:482] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:36:57.602226 30315 raft_consensus.cc:504] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:57.602707 30315 leader_election.cc:288] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349), a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
Aug 07 06:36:57 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084616, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.2@KRBTEST.COM
Aug 07 06:36:57 dist-test-slave-rwrg krb5kdc[27784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084616, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.3@KRBTEST.COM
I20260807 06:36:57.616655 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:57.602883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:44857 (local address 127.2.243.1:57111)
0807 06:36:57.603024 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:57.603036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:57.603098 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:57.604116 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:57.604117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:57.604540 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:57.604705 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:57.604711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.605826 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.605827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:57.606631 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:57.606634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.607270 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.607272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:57.607350 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:57.610778 (+ 3428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:57.610787 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:57.612897 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:57.615016 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:57.615016 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:57.615017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:57.615233 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:57.615486 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:57.615487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:57.615487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:57.615547 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:57.616006 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:57.616015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:57.616188 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 06:36:57.616311 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 06:36:57.616540 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":104,"thread_start_us":77,"threads_started":1}
I20260807 06:36:57.616664 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:57.603059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:57111 (local address 127.2.243.3:44857)
0807 06:36:57.603212 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:57.603221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:57.603239 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:36:57.603777 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:57.603781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:57.603877 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:57.604034 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:57.604850 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.605715 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.606722 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.607178 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.607299 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:57.613001 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:57.613004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:57.613005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:57.613014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:57.614944 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:57.615311 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:57.615314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:57.615314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:57.615422 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:57.615614 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:57.615615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:57.615615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:57.615928 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:57.616017 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:57.616396 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 06:36:57.616565 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260807 06:36:57.616675 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:57.602982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:54961 (local address 127.2.243.2:34349)
0807 06:36:57.603173 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:36:57.603194 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 06:36:57.603227 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:36:57.603789 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:36:57.603792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:36:57.603879 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:36:57.604009 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:36:57.604826 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.605695 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.606690 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.607125 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.607249 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:57.612556 (+ 5307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:36:57.612562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:36:57.612562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:36:57.612575 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:36:57.614682 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:57.615175 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:57.615177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:57.615178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:57.615293 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:36:57.615609 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:36:57.615610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:36:57.615610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:36:57.615928 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:36:57.616033 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 06:36:57.616396 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 06:36:57.616562 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":76,"threads_started":1}
I20260807 06:36:57.617202 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:57.602844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:34349 (local address 127.2.243.1:54961)
0807 06:36:57.602896 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:57.602909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:36:57.602965 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:36:57.604076 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:36:57.604079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:36:57.604471 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:36:57.604703 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:57.604709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.605801 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.605803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:57.606597 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:36:57.606599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:36:57.607219 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:36:57.607220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:36:57.607377 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:36:57.610812 (+ 3435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:36:57.610816 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:36:57.612408 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:36:57.614785 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:57.614792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:57.614794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:57.615086 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:57.615365 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:36:57.615366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:36:57.615366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:36:57.615508 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:36:57.616006 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:36:57.616015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:36:57.616188 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 06:36:57.616311 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 06:36:57.616642 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":167}
W20260807 06:36:57.617815 29675 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
W20260807 06:36:57.617893 29674 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:36:57.617923 29675 leader_election.cc:302] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a503483943f74de19ef8cb43cbb08d80; no voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0
I20260807 06:36:57.618083 30315 raft_consensus.cc:2726] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:36:57.710124 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:57.709838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:35663)
0807 06:36:57.710010 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:57.710075 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260807 06:36:58.494201 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:58.493963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:51157)
0807 06:36:58.494114 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:58.494155 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260807 06:36:58.576061 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:58.575853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:58453)
0807 06:36:58.575981 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:58.576021 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260807 06:36:58.711402 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:58.711171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:46761)
0807 06:36:58.711305 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:58.711358 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260807 06:36:59.119451 30328 raft_consensus.cc:482] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:36:59.119551 30328 raft_consensus.cc:504] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:36:59.119720 30328 leader_election.cc:288] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349), a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
W20260807 06:36:59.120200 29675 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
W20260807 06:36:59.120345 29674 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:36:59.120411 29674 leader_election.cc:302] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a503483943f74de19ef8cb43cbb08d80; no voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0
I20260807 06:36:59.120481 30328 raft_consensus.cc:2726] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:36:59.495002 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:59.494732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:51445)
0807 06:36:59.494900 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:59.494946 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260807 06:36:59.576889 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:59.576662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:59189)
0807 06:36:59.576811 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:59.576850 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260807 06:36:59.712230 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:36:59.711930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:57365)
0807 06:36:59.712115 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 06:36:59.712176 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
I20260807 06:37:00.495824 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:00.495576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:47699)
0807 06:37:00.495742 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:00.495781 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
I20260807 06:37:00.577629 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:00.577377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:46553)
0807 06:37:00.577548 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:00.577587 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":62,"threads_started":1}
I20260807 06:37:00.712980 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:00.712730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:57463)
0807 06:37:00.712898 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:00.712936 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260807 06:37:01.496774 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:01.496439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:57835)
0807 06:37:01.496685 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:01.496728 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":178,"threads_started":1}
I20260807 06:37:01.578384 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:01.578137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:35003)
0807 06:37:01.578291 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:01.578339 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260807 06:37:01.713752 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:01.713507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:59825)
0807 06:37:01.713668 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:01.713706 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260807 06:37:01.933104 30341 raft_consensus.cc:482] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:01.933224 30341 raft_consensus.cc:504] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:37:01.933434 30341 leader_election.cc:288] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349), a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
W20260807 06:37:01.933920 29675 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
W20260807 06:37:01.934077 29674 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:37:01.934146 29674 leader_election.cc:302] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a503483943f74de19ef8cb43cbb08d80; no voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0
I20260807 06:37:01.934234 30341 raft_consensus.cc:2726] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:37:02.497678 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:02.497305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:41231)
0807 06:37:02.497588 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:02.497627 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":236,"threads_started":1}
I20260807 06:37:02.579250 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:02.579011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:34269)
0807 06:37:02.579138 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:02.579199 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260807 06:37:02.714422 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:02.714207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:40367)
0807 06:37:02.714341 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:02.714379 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260807 06:37:03.498335 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:03.498111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:40793)
0807 06:37:03.498247 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:03.498281 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260807 06:37:03.579933 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:03.579669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:51085)
0807 06:37:03.579852 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:03.579889 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260807 06:37:03.715178 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:03.714924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:60891)
0807 06:37:03.715069 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:03.715129 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260807 06:37:04.499085 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:04.498835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:50725)
0807 06:37:04.498992 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:04.499034 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260807 06:37:04.580729 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:04.580413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:35497)
0807 06:37:04.580628 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:04.580670 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":61,"threads_started":1}
I20260807 06:37:04.715950 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:04.715761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:58065)
0807 06:37:04.715879 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:04.715911 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260807 06:37:05.499893 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:05.499645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:35125)
0807 06:37:05.499798 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:05.499843 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260807 06:37:05.581461 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:05.581232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:56821)
0807 06:37:05.581379 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:05.581417 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260807 06:37:05.716765 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:05.716449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:38593)
0807 06:37:05.716647 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:05.716705 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":118,"threads_started":1}
I20260807 06:37:06.500725 30362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:06.500394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:53567)
0807 06:37:06.500591 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:06.500659 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":68,"threads_started":1}
I20260807 06:37:06.582281 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:06.581974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:47973)
0807 06:37:06.582207 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:06.582240 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":180,"threads_started":1}
I20260807 06:37:06.600843 30365 raft_consensus.cc:482] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:06.600939 30365 raft_consensus.cc:504] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:37:06.601095 30365 leader_election.cc:288] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349), a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
W20260807 06:37:06.601539 29675 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
W20260807 06:37:06.601653 29674 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:37:06.601698 29674 leader_election.cc:302] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a503483943f74de19ef8cb43cbb08d80; no voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0
I20260807 06:37:06.601802 30365 raft_consensus.cc:2726] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:37:06.717461 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:06.717225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:58641)
0807 06:37:06.717358 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:06.717402 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260807 06:37:07.501479 30367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:07.501239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:48717)
0807 06:37:07.501389 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:07.501426 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260807 06:37:07.583140 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:07.582757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:37677)
0807 06:37:07.583033 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:07.583082 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":82,"threads_started":1}
I20260807 06:37:07.718189 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:07.717954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:40941)
0807 06:37:07.718114 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:07.718145 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260807 06:37:08.502267 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:08.502027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:57559)
0807 06:37:08.502181 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:08.502220 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260807 06:37:08.530368 30373 raft_consensus.cc:482] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:08.530465 30373 raft_consensus.cc:504] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d5be6ce3d2436d9fb544ec0a26c2f0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34349 } } peers { permanent_uuid: "a20d70a4fd134719ac07181a8b0fb133" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 44857 } } peers { permanent_uuid: "a503483943f74de19ef8cb43cbb08d80" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40043 } }
I20260807 06:37:08.530690 30373 leader_election.cc:288] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349), a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857)
W20260807 06:37:08.531067 29674 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b5d5be6ce3d2436d9fb544ec0a26c2f0 (127.2.243.2:34349): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
W20260807 06:37:08.531148 29675 leader_election.cc:341] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a20d70a4fd134719ac07181a8b0fb133 (127.2.243.3:44857): Not found: Tablet not found: 0f8e7a8111f04093be7b8f77c74e4c1c
I20260807 06:37:08.531224 29675 leader_election.cc:302] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a503483943f74de19ef8cb43cbb08d80; no voters: a20d70a4fd134719ac07181a8b0fb133, b5d5be6ce3d2436d9fb544ec0a26c2f0
I20260807 06:37:08.531309 30373 raft_consensus.cc:2726] T 0f8e7a8111f04093be7b8f77c74e4c1c P a503483943f74de19ef8cb43cbb08d80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:37:08.583918 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:08.583658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:49535)
0807 06:37:08.583824 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:08.583863 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
I20260807 06:37:08.718888 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:08.718674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:51023)
0807 06:37:08.718803 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:08.718842 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260807 06:37:09.503105 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:09.502862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:56287)
0807 06:37:09.502985 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:09.503037 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260807 06:37:09.584610 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:09.584352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:57289)
0807 06:37:09.584513 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:09.584549 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260807 06:37:09.719630 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:09.719379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:42049)
0807 06:37:09.719515 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:09.719587 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260807 06:37:10.503871 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:10.503595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:59911)
0807 06:37:10.503784 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:10.503814 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
I20260807 06:37:10.585289 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:10.585057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:60105)
0807 06:37:10.585204 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:10.585244 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260807 06:37:10.720338 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:10.720134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:34579)
0807 06:37:10.720259 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:10.720295 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260807 06:37:11.504709 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:11.504425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.1:36495)
0807 06:37:11.504632 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:11.504667 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":153,"threads_started":1}
I20260807 06:37:11.585983 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:11.585773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.2:35557)
0807 06:37:11.585906 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:11.585937 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260807 06:37:11.720304 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 29592
I20260807 06:37:11.725081 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:11.724882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39945 (local address 127.2.243.3:45259)
0807 06:37:11.724990 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:11.725024 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.62:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260807 06:37:11.734627 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 29866
I20260807 06:37:11.747049 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 30086
2026-08-07T06:37:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58622 ms)
[----------] 30 tests from SecurityITest (239276 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:11 dist-test-slave-rwrg krb5kdc[30395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:11 dist-test-slave-rwrg krb5kdc[30395](info): set up 2 sockets
Aug 07 06:37:11 dist-test-slave-rwrg krb5kdc[30395](info): commencing operation
krb5kdc: starting...
W20260807 06:37:13.061352 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:13.183561 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42881
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:13.264873 30414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:13.265136 30414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:13.265188 30414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:13.266319 30414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:13.266388 30414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:13.266417 30414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:13.266448 30414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:13.266472 30414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:13.267946 30414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:13.268240 30414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:13.268509 30414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:13.273865 30419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:13.273875 30420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:13.273866 30422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:13.273926 30414 server_base.cc:1034] running on GCE node
I20260807 06:37:13.274312 30414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:13.274549 30414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:13.275686 30414 hybrid_clock.cc:648] HybridClock initialized: now 1786084633275672 us; error 29 us; skew 500 ppm
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:13.278165 30414 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:13.278630 30414 webserver.cc:460] Webserver started at http://127.2.243.62:41297/ using document root <none> and password file <none>
I20260807 06:37:13.278870 30414 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:13.278939 30414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:13.279124 30414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:13.280203 30414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "2bc7c4987f4b4b619c796830f6287a60"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "eff9c904e5811fb9ce5f96bd8a77f381"
server_key_iv: "4211e92aef0a2a07ec68dd578bb913ff"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.280634 30414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "2bc7c4987f4b4b619c796830f6287a60"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "eff9c904e5811fb9ce5f96bd8a77f381"
server_key_iv: "4211e92aef0a2a07ec68dd578bb913ff"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.281993 30414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:13.282778 30437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.282946 30414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:13.283056 30414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "2bc7c4987f4b4b619c796830f6287a60"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "eff9c904e5811fb9ce5f96bd8a77f381"
server_key_iv: "4211e92aef0a2a07ec68dd578bb913ff"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.283145 30414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:13.298106 30414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:13.299998 30414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:13.300189 30414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:13.304339 30414 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42881
I20260807 06:37:13.304383 30512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42881 every 8 connection(s)
I20260807 06:37:13.304862 30414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:13.305730 30513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:13.307928 30513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60: Bootstrap starting.
I20260807 06:37:13.308583 30513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:13.308950 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 30414
I20260807 06:37:13.308948 30513 log.cc:824] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:13.309056 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:13.309284 3020 external_mini_cluster.cc:1428] Setting key c5d3e32ecfab3593e475bc97a05dd9ab
I20260807 06:37:13.310094 30513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60: No bootstrap required, opened a new log
I20260807 06:37:13.312624 30513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc7c4987f4b4b619c796830f6287a60" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42881 } }
I20260807 06:37:13.312793 30513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:13.312830 30513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bc7c4987f4b4b619c796830f6287a60, State: Initialized, Role: FOLLOWER
I20260807 06:37:13.313015 30513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc7c4987f4b4b619c796830f6287a60" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42881 } }
I20260807 06:37:13.313117 30513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:13.313161 30513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:13.313208 30513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:13.313776 30513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc7c4987f4b4b619c796830f6287a60" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42881 } }
I20260807 06:37:13.313899 30513 leader_election.cc:302] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bc7c4987f4b4b619c796830f6287a60; no voters:
I20260807 06:37:13.314049 30513 leader_election.cc:288] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:13.314128 30522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:13.314288 30513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:13.314332 30522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [term 1 LEADER]: Becoming Leader. State: Replica: 2bc7c4987f4b4b619c796830f6287a60, State: Running, Role: LEADER
I20260807 06:37:13.314610 30522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc7c4987f4b4b619c796830f6287a60" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42881 } }
I20260807 06:37:13.314992 30523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bc7c4987f4b4b619c796830f6287a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc7c4987f4b4b619c796830f6287a60" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42881 } } }
I20260807 06:37:13.315069 30523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:13.315323 30525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bc7c4987f4b4b619c796830f6287a60. Latest consensus state: current_term: 1 leader_uuid: "2bc7c4987f4b4b619c796830f6287a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc7c4987f4b4b619c796830f6287a60" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42881 } } }
I20260807 06:37:13.315393 30525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:13.315834 30538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:13.316370 30538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:13.317960 30538 catalog_manager.cc:1347] Generated new cluster ID: 4170e4ef02774a9b98c61dda1fb7e84b
I20260807 06:37:13.318012 30538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:13.321482 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.309821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.2.243.62:42881)
0807 06:37:13.310024 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.310033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.310070 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:37:13.310583 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.310591 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.310648 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.310792 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:13.311381 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.312797 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.313666 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.314329 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.315977 (+ 1648us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.317840 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:13.317843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:13.317843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:13.317855 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:13.319831 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.320248 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.320250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.320251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.320369 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.320574 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.320575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.320575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.320894 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:13.321061 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.321141 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.321276 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260807 06:37:13.324889 30538 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:13.325354 30538 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:13.333874 30538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60: Generated new TSK 0
I20260807 06:37:13.334086 30538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:37:13.361723 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42881
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:13.446908 30558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:13.447150 30558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:13.447192 30558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:13.448349 30558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:13.448416 30558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:13.448462 30558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:37:13.449999 30558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:13.450284 30558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:13.450522 30558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:13.455549 30576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:13.455549 30570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:13.455556 30569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:13.455905 30558 server_base.cc:1034] running on GCE node
I20260807 06:37:13.456164 30558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:13.456410 30558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:13.457580 30558 hybrid_clock.cc:648] HybridClock initialized: now 1786084633457564 us; error 69 us; skew 500 ppm
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:13.460253 30558 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:37:13.460808 30558 webserver.cc:460] Webserver started at http://127.2.243.1:37189/ using document root <none> and password file <none>
I20260807 06:37:13.461063 30558 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:13.461135 30558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:13.461289 30558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:13.462328 30558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "d5d77481da1146f28b898af782128a1a"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "ba7dcc90f54666729cb94e25533d27c3"
server_key_iv: "d9b1afbdcf93ea2aada507aea4b67cb9"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.462779 30558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "d5d77481da1146f28b898af782128a1a"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "ba7dcc90f54666729cb94e25533d27c3"
server_key_iv: "d9b1afbdcf93ea2aada507aea4b67cb9"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.464148 30558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:13.464998 30592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.465193 30558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:37:13.465301 30558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "d5d77481da1146f28b898af782128a1a"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "ba7dcc90f54666729cb94e25533d27c3"
server_key_iv: "d9b1afbdcf93ea2aada507aea4b67cb9"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.465374 30558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:13.477341 30558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:13.479708 30558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:13.479873 30558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:13.480106 30558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:13.480422 30558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:13.480476 30558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.480537 30558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:13.480566 30558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.487066 30558 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:34515
I20260807 06:37:13.487109 30769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:34515 every 8 connection(s)
I20260807 06:37:13.487520 30558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:13.497423 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 30558
I20260807 06:37:13.497555 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:37:13.497639 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.488328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:60613 (local address 127.2.243.62:42881)
0807 06:37:13.488376 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.488379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.488695 (+ 316us) server_negotiation.cc:407] Connection header received
0807 06:37:13.489415 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.489416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.489430 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.489499 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:13.490510 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.491040 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.492051 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.492267 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.492345 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.494636 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:13.494638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:13.494638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:13.494645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:13.496004 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.496446 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.496448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.496448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.496556 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.496824 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.496826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.496826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.496948 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:13.497047 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.497373 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.497455 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:37:13.497773 3020 external_mini_cluster.cc:1428] Setting key 9057e6badf6c4c58b693640f79170de9
I20260807 06:37:13.497836 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.488331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42881 (local address 127.2.243.1:60613)
0807 06:37:13.488540 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:13.488584 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:13.489289 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:13.489578 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:13.489582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:13.490188 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:13.490380 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.490385 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.491148 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.491149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.491942 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.491944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.492366 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.492368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.492523 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.493071 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:13.493079 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:13.494456 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:13.496127 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:13.496134 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:13.496136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:13.496364 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:13.496639 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:13.496640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:13.496640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:13.496749 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:13.497042 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:13.497050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:13.497194 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 06:37:13.497332 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 06:37:13.497520 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260807 06:37:13.498469 30770 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42881
I20260807 06:37:13.498591 30770 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:13.498790 30770 heartbeater.cc:511] Master 127.2.243.62:42881 requested a full tablet report, sending...
I20260807 06:37:13.499361 30465 ts_manager.cc:194] Registered new tserver with Master: d5d77481da1146f28b898af782128a1a (127.2.243.1:34515)
I20260807 06:37:13.500007 30465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:60613
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:37:13.523809 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42881
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:13.606616 30776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:13.606827 30776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:13.606853 30776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:13.608008 30776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:13.608058 30776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:13.608085 30776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:37:13.609596 30776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:13.609841 30776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:13.610098 30776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:13.615114 30783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:13.615092 30782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:13.615348 30776 server_base.cc:1034] running on GCE node
W20260807 06:37:13.615468 30785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:13.615742 30776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:13.616011 30776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:13.617164 30776 hybrid_clock.cc:648] HybridClock initialized: now 1786084633617151 us; error 31 us; skew 500 ppm
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:13.619654 30776 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:37:13.620102 30776 webserver.cc:460] Webserver started at http://127.2.243.2:37851/ using document root <none> and password file <none>
I20260807 06:37:13.620338 30776 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:13.620404 30776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:13.620606 30776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:13.621608 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "8dda6707bc704e6ea48fb3f29ba8fce0"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "14682d3c2aa68be3491b538de8d2ecaa"
server_key_iv: "6273af489df0eb2dd46f7a8f652b7249"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.621951 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "8dda6707bc704e6ea48fb3f29ba8fce0"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "14682d3c2aa68be3491b538de8d2ecaa"
server_key_iv: "6273af489df0eb2dd46f7a8f652b7249"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.623253 30776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:13.624050 30795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.624239 30776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:13.624338 30776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "8dda6707bc704e6ea48fb3f29ba8fce0"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "14682d3c2aa68be3491b538de8d2ecaa"
server_key_iv: "6273af489df0eb2dd46f7a8f652b7249"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.624418 30776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:13.647527 30776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:13.649677 30776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:13.649866 30776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:13.650125 30776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:13.650453 30776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:13.650509 30776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.650552 30776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:13.650579 30776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.656735 30776 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:38035
I20260807 06:37:13.656790 30981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:38035 every 8 connection(s)
I20260807 06:37:13.657117 30776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:37:13.659112 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 30776
I20260807 06:37:13.659220 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:37:13.659433 3020 external_mini_cluster.cc:1428] Setting key 3e420716008ca1c9633179a7c2f8c680
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:13.667644 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.657889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:53739 (local address 127.2.243.62:42881)
0807 06:37:13.657937 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.657939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.658176 (+ 237us) server_negotiation.cc:407] Connection header received
0807 06:37:13.658738 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.658740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.658753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.658810 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:13.659942 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.660437 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.661545 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.661877 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.661952 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.664327 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:13.664329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:13.664329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:13.664337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:13.666041 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.666516 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.666517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.666517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.666592 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.666859 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.666860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.666861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.667031 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:13.667110 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.667418 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.667514 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:37:13.667765 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.657849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42881 (local address 127.2.243.2:53739)
0807 06:37:13.658048 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:13.658086 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:13.658624 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:13.658888 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:13.658892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:13.659638 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:13.659839 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.659845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.660587 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.660588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.661434 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.661437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.661959 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.661960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.662076 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.662622 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:13.662630 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:13.664225 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:13.666151 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:13.666158 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:13.666160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:13.666428 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:13.666671 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:13.666672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:13.666672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:13.666783 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:13.667111 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:13.667116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:13.667269 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 06:37:13.667365 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 06:37:13.667500 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
I20260807 06:37:13.668341 30984 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42881
I20260807 06:37:13.668462 30984 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:13.668715 30984 heartbeater.cc:511] Master 127.2.243.62:42881 requested a full tablet report, sending...
I20260807 06:37:13.669247 30465 ts_manager.cc:194] Registered new tserver with Master: 8dda6707bc704e6ea48fb3f29ba8fce0 (127.2.243.2:38035)
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
I20260807 06:37:13.669804 30465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:53739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:37:13.687463 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42881
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:13.773213 30992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:13.773412 30992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:13.773434 30992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:13.774528 30992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:13.774569 30992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:13.774633 30992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:37:13.776010 30992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:13.776208 30992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:13.776376 30992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:13.781221 31003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:13.781292 31004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:13.781224 31007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:13.781683 30992 server_base.cc:1034] running on GCE node
I20260807 06:37:13.781875 30992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:13.782116 30992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:13.783279 30992 hybrid_clock.cc:648] HybridClock initialized: now 1786084633783261 us; error 40 us; skew 500 ppm
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:13.786113 30992 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:37:13.786672 30992 webserver.cc:460] Webserver started at http://127.2.243.3:40309/ using document root <none> and password file <none>
I20260807 06:37:13.786865 30992 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:13.786908 30992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:13.787029 30992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:13.788089 30992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "84e76fb0cab94a7ca99b03225cb0a38f"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "72c6f0ec1f677947ed84286421bc5606"
server_key_iv: "3c6642d359622d169e9c894e416ceff6"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.788403 30992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "84e76fb0cab94a7ca99b03225cb0a38f"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "72c6f0ec1f677947ed84286421bc5606"
server_key_iv: "3c6642d359622d169e9c894e416ceff6"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.789877 30992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:13.790623 31018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.790794 30992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:37:13.790871 30992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "84e76fb0cab94a7ca99b03225cb0a38f"
format_stamp: "Formatted at 2026-08-07 06:37:13 on dist-test-slave-rwrg"
server_key: "72c6f0ec1f677947ed84286421bc5606"
server_key_iv: "3c6642d359622d169e9c894e416ceff6"
server_key_version: "encryptionkey@0"
I20260807 06:37:13.790923 30992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:13.811967 30992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:13.813977 30992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:13.814106 30992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:13.814304 30992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:13.814769 30992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:13.814824 30992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.814875 30992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:13.814908 30992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:13.821674 30992 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:37513
I20260807 06:37:13.821753 31198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:37513 every 8 connection(s)
I20260807 06:37:13.822068 30992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:37:13.822809 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 30992
I20260807 06:37:13.822902 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:37:13.823087 3020 external_mini_cluster.cc:1428] Setting key 58ecdac6354d536dc7ae024e0b967c2c
Aug 07 06:37:13 dist-test-slave-rwrg krb5kdc[30395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084633, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:13.831876 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.822836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:56365 (local address 127.2.243.62:42881)
0807 06:37:13.822878 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.822880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.823117 (+ 237us) server_negotiation.cc:407] Connection header received
0807 06:37:13.823876 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.823891 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.823905 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.823958 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:13.825241 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.825733 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.826789 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.826966 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.827017 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.829290 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:13.829291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:13.829292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:13.829298 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:13.830704 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.831092 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.831093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.831093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.831146 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.831368 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.831369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.831369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.831465 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:13.831514 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.831755 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.831813 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:37:13.832134 31206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.822816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42881 (local address 127.2.243.3:56365)
0807 06:37:13.822987 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:13.823030 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:13.823756 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:13.824064 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:13.824069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:13.824886 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:13.825133 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.825138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.825820 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.825821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.826711 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.826713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.827049 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.827050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.827150 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.827628 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:13.827636 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:13.829193 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:13.830784 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:13.830790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:13.830792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:13.831025 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:13.831213 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:13.831213 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:13.831214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:13.831313 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:13.831524 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:13.831528 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:13.831643 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 06:37:13.831730 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:37:13.831848 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260807 06:37:13.832787 31199 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42881
I20260807 06:37:13.832891 31199 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:13.833067 31199 heartbeater.cc:511] Master 127.2.243.62:42881 requested a full tablet report, sending...
I20260807 06:37:13.833438 30465 ts_manager.cc:194] Registered new tserver with Master: 84e76fb0cab94a7ca99b03225cb0a38f (127.2.243.3:37513)
I20260807 06:37:13.833882 30465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:56365
I20260807 06:37:13.834833 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:37:13.842546 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.836196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.2.243.62:42881)
0807 06:37:13.836264 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.836284 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.836395 (+ 111us) server_negotiation.cc:407] Connection header received
0807 06:37:13.836481 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.836482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.836521 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.836575 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:13.837182 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.837692 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.838510 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.838743 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.838827 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.839996 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:13.839997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:13.839998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:13.840014 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:13.841245 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.841621 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.841622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.841622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.841685 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:13.841918 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:13.841919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:13.841919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:13.842087 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:13.842208 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.842351 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.842436 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:37:13.844120 30465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:37:13.846024 30465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:37:13.853084 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.848567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:34515 (local address 127.0.0.1:38642)
0807 06:37:13.848756 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:13.848801 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:13.848875 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:13.849257 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:13.849261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:13.849264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:13.849360 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.849364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.850770 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.850771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.851887 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.851889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.852756 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.852758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.852852 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.852867 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:37:13.852949 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 06:37:13.853002 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":112,"thread_start_us":85,"threads_started":1}
I20260807 06:37:13.853223 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.848814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.2.243.3:37513)
0807 06:37:13.849385 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.849389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.849399 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:13.849432 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.849434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.849448 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.849531 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:13.849734 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.850603 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.852199 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.852871 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.852960 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.853012 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.853103 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.853152 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":87,"threads_started":1}
I20260807 06:37:13.853472 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.848833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.2.243.1:34515)
0807 06:37:13.849021 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.849025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.849036 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:37:13.849076 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.849078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.849095 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.849213 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:13.849448 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.850643 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.851981 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.852671 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.852745 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.852803 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.853263 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.853367 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
I20260807 06:37:13.853392 31220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.848567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:37513 (local address 127.0.0.1:46840)
0807 06:37:13.848979 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:13.848991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:13.849048 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:13.849588 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:13.849589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:13.849591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:13.849639 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.849643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.850722 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.850723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.852107 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.852110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.852963 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.852964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.853027 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.853032 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:37:13.853089 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:37:13.853133 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":39,"thread_start_us":75,"threads_started":1}
I20260807 06:37:13.853765 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.848782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.2.243.2:38035)
0807 06:37:13.849687 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:13.849692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:13.849702 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:13.849739 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:13.849741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:13.849755 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:13.849837 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:13.849961 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.850987 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.852369 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.853117 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.853191 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.853255 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:13.853604 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 06:37:13.853653 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":90,"threads_started":1}
I20260807 06:37:13.853538 31222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:13.848602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:38035 (local address 127.0.0.1:52232)
0807 06:37:13.849120 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:13.849129 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:13.849167 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:13.849848 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:13.849849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:13.849851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:13.849884 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.849886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.851090 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.851091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.852273 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:13.852276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:13.853294 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:13.853295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:13.853365 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:13.853369 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:37:13.853421 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:37:13.853458 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":91,"threads_started":1}
I20260807 06:37:13.854463 30657 tablet_service.cc:1467] Processing CreateTablet for tablet 11cd11e1bf974685898722dc0599d84a (DEFAULT_TABLE table=test-table [id=c93f1eaf4c634780a281d33aec9c60ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:13.854542 31100 tablet_service.cc:1467] Processing CreateTablet for tablet 11cd11e1bf974685898722dc0599d84a (DEFAULT_TABLE table=test-table [id=c93f1eaf4c634780a281d33aec9c60ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:13.854827 31100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cd11e1bf974685898722dc0599d84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:13.854833 30657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cd11e1bf974685898722dc0599d84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:13.857533 31231 tablet_bootstrap.cc:492] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a: Bootstrap starting.
I20260807 06:37:13.857628 31232 tablet_bootstrap.cc:492] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Bootstrap starting.
I20260807 06:37:13.858019 30877 tablet_service.cc:1467] Processing CreateTablet for tablet 11cd11e1bf974685898722dc0599d84a (DEFAULT_TABLE table=test-table [id=c93f1eaf4c634780a281d33aec9c60ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:13.858335 31232 tablet_bootstrap.cc:654] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:13.858364 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cd11e1bf974685898722dc0599d84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:13.858414 31231 tablet_bootstrap.cc:654] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:13.858840 31232 log.cc:824] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:13.858942 31231 log.cc:824] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:13.859670 31231 tablet_bootstrap.cc:492] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a: No bootstrap required, opened a new log
I20260807 06:37:13.859747 31231 ts_tablet_manager.cc:1397] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:37:13.861202 31232 tablet_bootstrap.cc:492] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: No bootstrap required, opened a new log
I20260807 06:37:13.861325 31232 ts_tablet_manager.cc:1397] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:37:13.861654 31237 tablet_bootstrap.cc:492] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Bootstrap starting.
I20260807 06:37:13.861821 31231 raft_consensus.cc:348] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:13.862016 31231 raft_consensus.cc:374] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:13.862061 31231 raft_consensus.cc:729] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d77481da1146f28b898af782128a1a, State: Initialized, Role: FOLLOWER
I20260807 06:37:13.862203 31231 consensus_queue.cc:260] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:13.862453 31231 ts_tablet_manager.cc:1428] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:37:13.862507 31237 tablet_bootstrap.cc:654] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:13.862596 30770 heartbeater.cc:503] Master 127.2.243.62:42881 was elected leader, sending a full tablet report...
I20260807 06:37:13.863193 31237 log.cc:824] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:13.863916 31232 raft_consensus.cc:348] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:13.864084 31237 tablet_bootstrap.cc:492] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: No bootstrap required, opened a new log
I20260807 06:37:13.864132 31232 raft_consensus.cc:374] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:13.864159 31237 ts_tablet_manager.cc:1397] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:37:13.864178 31232 raft_consensus.cc:729] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e76fb0cab94a7ca99b03225cb0a38f, State: Initialized, Role: FOLLOWER
I20260807 06:37:13.864342 31232 consensus_queue.cc:260] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:13.864737 31232 ts_tablet_manager.cc:1428] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 06:37:13.864811 31199 heartbeater.cc:503] Master 127.2.243.62:42881 was elected leader, sending a full tablet report...
I20260807 06:37:13.866433 31237 raft_consensus.cc:348] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:13.866585 31237 raft_consensus.cc:374] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:13.866612 31237 raft_consensus.cc:729] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dda6707bc704e6ea48fb3f29ba8fce0, State: Initialized, Role: FOLLOWER
I20260807 06:37:13.866776 31237 consensus_queue.cc:260] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:13.867108 31237 ts_tablet_manager.cc:1428] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:37:13.867129 30984 heartbeater.cc:503] Master 127.2.243.62:42881 was elected leader, sending a full tablet report...
W20260807 06:37:13.907662 30986 tablet.cc:2369] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:37:13.988283 30772 tablet.cc:2369] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:37:14.072698 31205 tablet.cc:2369] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:37:14.159307 31242 raft_consensus.cc:482] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:14.159433 31242 raft_consensus.cc:504] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:14.159868 31242 leader_election.cc:288] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dda6707bc704e6ea48fb3f29ba8fce0 (127.2.243.2:38035), d5d77481da1146f28b898af782128a1a (127.2.243.1:34515)
I20260807 06:37:14.163941 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.160110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:36857 (local address 127.2.243.2:38035)
0807 06:37:14.160202 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.160204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.160368 (+ 164us) server_negotiation.cc:407] Connection header received
0807 06:37:14.160455 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.160456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.160469 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.160618 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:14.160746 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.161421 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.162995 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.163585 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.163650 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.163658 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.163828 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.163868 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 06:37:14.163942 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.160158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:48619 (local address 127.2.243.1:34515)
0807 06:37:14.160217 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.160219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.160229 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:14.160309 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.160310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.160328 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.160395 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:14.160617 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.161398 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.162995 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.163564 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.163647 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.163658 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.163830 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.163867 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:37:14.163968 31206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.160056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:34515 (local address 127.2.243.3:48619)
0807 06:37:14.160163 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.160175 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.160229 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.160400 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.160401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.160403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:14.160480 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.160482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.161503 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.161504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.162882 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.162885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.163661 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.163662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.163746 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.163748 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.163808 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.163863 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":14}
I20260807 06:37:14.163975 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.160079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:38035 (local address 127.2.243.3:36857)
0807 06:37:14.160276 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.160285 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.160383 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.160628 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.160629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.160630 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:14.160680 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.160681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.161503 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.161504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.162882 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.162885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.163661 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.163662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.163743 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.163748 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.163808 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.163863 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":164,"thread_start_us":122,"threads_started":1}
I20260807 06:37:14.164381 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd11e1bf974685898722dc0599d84a" candidate_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d77481da1146f28b898af782128a1a" is_pre_election: true
I20260807 06:37:14.164449 30908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd11e1bf974685898722dc0599d84a" candidate_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" is_pre_election: true
I20260807 06:37:14.164605 30908 raft_consensus.cc:2454] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e76fb0cab94a7ca99b03225cb0a38f in term 0.
I20260807 06:37:14.164597 30684 raft_consensus.cc:2454] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e76fb0cab94a7ca99b03225cb0a38f in term 0.
I20260807 06:37:14.164909 31019 leader_election.cc:302] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e76fb0cab94a7ca99b03225cb0a38f, d5d77481da1146f28b898af782128a1a; no voters:
I20260807 06:37:14.165055 31242 raft_consensus.cc:2781] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:37:14.165110 31242 raft_consensus.cc:482] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:37:14.165135 31242 raft_consensus.cc:3037] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:14.165815 31242 raft_consensus.cc:504] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:14.165957 31242 leader_election.cc:288] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [CANDIDATE]: Term 1 election: Requested vote from peers 8dda6707bc704e6ea48fb3f29ba8fce0 (127.2.243.2:38035), d5d77481da1146f28b898af782128a1a (127.2.243.1:34515)
I20260807 06:37:14.166136 30908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd11e1bf974685898722dc0599d84a" candidate_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0"
I20260807 06:37:14.166121 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cd11e1bf974685898722dc0599d84a" candidate_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d77481da1146f28b898af782128a1a"
I20260807 06:37:14.166204 30908 raft_consensus.cc:3037] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:14.166225 30684 raft_consensus.cc:3037] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:14.166950 30908 raft_consensus.cc:2454] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e76fb0cab94a7ca99b03225cb0a38f in term 1.
I20260807 06:37:14.166954 30684 raft_consensus.cc:2454] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e76fb0cab94a7ca99b03225cb0a38f in term 1.
I20260807 06:37:14.167168 31027 leader_election.cc:302] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e76fb0cab94a7ca99b03225cb0a38f, 8dda6707bc704e6ea48fb3f29ba8fce0; no voters:
I20260807 06:37:14.167285 31242 raft_consensus.cc:2781] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:14.167425 31242 raft_consensus.cc:686] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 1 LEADER]: Becoming Leader. State: Replica: 84e76fb0cab94a7ca99b03225cb0a38f, State: Running, Role: LEADER
I20260807 06:37:14.167594 31242 consensus_queue.cc:237] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } }
I20260807 06:37:14.168577 30465 catalog_manager.cc:5495] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e76fb0cab94a7ca99b03225cb0a38f (127.2.243.3). New cstate: current_term: 1 leader_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e76fb0cab94a7ca99b03225cb0a38f" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37513 } health_report { overall_health: HEALTHY } } }
I20260807 06:37:14.204803 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.202302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.2.243.3:37513)
0807 06:37:14.202375 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.202377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.202385 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:37:14.202426 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.202427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.202446 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.202563 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:14.202673 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.203129 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.203913 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.204111 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.204191 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.204352 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:14.204468 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:14.204644 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.204665 (+ 21us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.204685 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:37:14.206871 30908 raft_consensus.cc:1261] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 1 FOLLOWER]: Refusing update from remote peer 84e76fb0cab94a7ca99b03225cb0a38f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:14.206889 30684 raft_consensus.cc:1261] T 11cd11e1bf974685898722dc0599d84a P d5d77481da1146f28b898af782128a1a [term 1 FOLLOWER]: Refusing update from remote peer 84e76fb0cab94a7ca99b03225cb0a38f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:14.207285 31247 consensus_queue.cc:1035] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dda6707bc704e6ea48fb3f29ba8fce0" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 38035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:14.207430 31242 consensus_queue.cc:1035] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d77481da1146f28b898af782128a1a" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:14.210332 31251 mvcc.cc:205] Tried to move back new op lower bound from 7315802661708099584 to 7315802661551067136. Current Snapshot: MvccSnapshot[applied={T|T < 7315802661708099584}]
I20260807 06:37:14.210577 31257 mvcc.cc:205] Tried to move back new op lower bound from 7315802661708099584 to 7315802661551067136. Current Snapshot: MvccSnapshot[applied={T|T < 7315802661708099584}]
I20260807 06:37:14.212811 30464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:14.212913 30464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:14.213804 30464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bc7c4987f4b4b619c796830f6287a60: Sending DeleteTablet for 3 replicas of tablet 11cd11e1bf974685898722dc0599d84a
I20260807 06:37:14.214193 30657 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd11e1bf974685898722dc0599d84a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:14 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:38642
I20260807 06:37:14.214427 30877 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd11e1bf974685898722dc0599d84a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:14 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:52232
I20260807 06:37:14.214640 31100 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cd11e1bf974685898722dc0599d84a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:14 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:46840
I20260807 06:37:14.214716 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 30558
I20260807 06:37:14.219096 31267 tablet_replica.cc:330] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: stopping tablet replica
I20260807 06:37:14.219204 31267 raft_consensus.cc:2229] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:14.219306 31267 raft_consensus.cc:2258] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:14.226985 31267 ts_tablet_manager.cc:1905] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:14.228645 31268 tablet_replica.cc:330] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: stopping tablet replica
I20260807 06:37:14.228940 31268 raft_consensus.cc:2229] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:37:14.229311 31268 raft_consensus.cc:2258] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:14.230882 31268 ts_tablet_manager.cc:1905] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:14.232625 31267 ts_tablet_manager.cc:1918] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:14.232766 31267 log.cc:1196] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/11cd11e1bf974685898722dc0599d84a
I20260807 06:37:14.233111 31267 ts_tablet_manager.cc:1939] T 11cd11e1bf974685898722dc0599d84a P 8dda6707bc704e6ea48fb3f29ba8fce0: Deleting consensus metadata
I20260807 06:37:14.233656 30444 catalog_manager.cc:4841] TS 8dda6707bc704e6ea48fb3f29ba8fce0 (127.2.243.2:38035): tablet 11cd11e1bf974685898722dc0599d84a (table test-table [id=c93f1eaf4c634780a281d33aec9c60ea]) successfully deleted
I20260807 06:37:14.233734 31268 ts_tablet_manager.cc:1918] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:14.233922 31268 log.cc:1196] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/11cd11e1bf974685898722dc0599d84a
I20260807 06:37:14.234414 31268 ts_tablet_manager.cc:1939] T 11cd11e1bf974685898722dc0599d84a P 84e76fb0cab94a7ca99b03225cb0a38f: Deleting consensus metadata
I20260807 06:37:14.235038 30441 catalog_manager.cc:4841] TS 84e76fb0cab94a7ca99b03225cb0a38f (127.2.243.3:37513): tablet 11cd11e1bf974685898722dc0599d84a (table test-table [id=c93f1eaf4c634780a281d33aec9c60ea]) successfully deleted
W20260807 06:37:14.242309 30439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:34515 (error 108)
I20260807 06:37:14.242522 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 30776
I20260807 06:37:14.243068 31222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.242728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:34515 (local address 127.0.0.1:38656)
0807 06:37:14.242777 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.243029 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 06:37:14.245684 30439 catalog_manager.cc:4584] TS d5d77481da1146f28b898af782128a1a (127.2.243.1:34515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 11cd11e1bf974685898722dc0599d84a: Network error: Client connection negotiation failed: client connection to 127.2.243.1:34515: connect: Connection refused (error 111)
I20260807 06:37:14.255362 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 30992
I20260807 06:37:14.263190 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 30414
2026-08-07T06:37:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2514 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T06:37:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:14Z Disabled control of system clock
I20260807 06:37:14.296032 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42651
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:42651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:14.378469 31277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:14.378705 31277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:14.378726 31277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:14.379848 31277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:14.379882 31277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:14.379895 31277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:14.379907 31277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:14.379920 31277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:14.381361 31277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:42651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:42651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:14.381572 31277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:14.381815 31277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:37:14.386879 31277 server_base.cc:1034] running on GCE node
W20260807 06:37:14.386834 31283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.386847 31286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.386885 31284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:14.387423 31277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:14.387701 31277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:14.388849 31277 hybrid_clock.cc:648] HybridClock initialized: now 1786084634388838 us; error 29 us; skew 500 ppm
I20260807 06:37:14.390195 31277 webserver.cc:460] Webserver started at http://127.2.243.62:34427/ using document root <none> and password file <none>
I20260807 06:37:14.390471 31277 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:14.390586 31277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:14.390764 31277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:14.391849 31277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "b0cad99e49814cb4b53e679e29c028e0"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "a662deb341c37de980be4cf82c4580c3"
server_key_iv: "8d3065de67b5baa9401c1a7f87e74c99"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.392237 31277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "b0cad99e49814cb4b53e679e29c028e0"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "a662deb341c37de980be4cf82c4580c3"
server_key_iv: "8d3065de67b5baa9401c1a7f87e74c99"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.393723 31277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:14.394610 31303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.394806 31277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:14.394901 31277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "b0cad99e49814cb4b53e679e29c028e0"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "a662deb341c37de980be4cf82c4580c3"
server_key_iv: "8d3065de67b5baa9401c1a7f87e74c99"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.394986 31277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:14.421576 31277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:14.421962 31277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:14.422132 31277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:14.426431 31277 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:42651
I20260807 06:37:14.426488 31389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:42651 every 8 connection(s)
I20260807 06:37:14.426848 31277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:14.427886 31390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:14.430186 31390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0: Bootstrap starting.
I20260807 06:37:14.430740 31390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:14.431010 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 31277
I20260807 06:37:14.431103 31390 log.cc:824] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:14.431136 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:14.431336 3020 external_mini_cluster.cc:1428] Setting key 8c48f4996be957c3aa9466d2066faae9
I20260807 06:37:14.431919 31390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0: No bootstrap required, opened a new log
I20260807 06:37:14.434729 31390 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cad99e49814cb4b53e679e29c028e0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42651 } }
I20260807 06:37:14.434887 31390 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:14.434916 31390 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0cad99e49814cb4b53e679e29c028e0, State: Initialized, Role: FOLLOWER
I20260807 06:37:14.435022 31390 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cad99e49814cb4b53e679e29c028e0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42651 } }
I20260807 06:37:14.435104 31390 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:14.435141 31390 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:14.435194 31390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:14.435763 31390 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cad99e49814cb4b53e679e29c028e0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42651 } }
I20260807 06:37:14.435856 31390 leader_election.cc:302] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0cad99e49814cb4b53e679e29c028e0; no voters:
I20260807 06:37:14.436100 31390 leader_election.cc:288] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:14.436126 31396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:14.436240 31396 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [term 1 LEADER]: Becoming Leader. State: Replica: b0cad99e49814cb4b53e679e29c028e0, State: Running, Role: LEADER
I20260807 06:37:14.436372 31390 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:14.436383 31396 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cad99e49814cb4b53e679e29c028e0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42651 } }
I20260807 06:37:14.436884 31399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad99e49814cb4b53e679e29c028e0. Latest consensus state: current_term: 1 leader_uuid: "b0cad99e49814cb4b53e679e29c028e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cad99e49814cb4b53e679e29c028e0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42651 } } }
I20260807 06:37:14.437130 31399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:14.437040 31397 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0cad99e49814cb4b53e679e29c028e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cad99e49814cb4b53e679e29c028e0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 42651 } } }
I20260807 06:37:14.437359 31397 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:14.437553 31409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:14.438105 31409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:14.438454 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.431881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.2.243.62:42651)
0807 06:37:14.432112 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.432121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.432154 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:37:14.433194 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.433204 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.433279 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.433434 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:14.433657 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.435075 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.436047 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.436708 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.436848 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.437628 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:14.437633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:14.437633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:14.437844 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:14.437888 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:14.437904 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:14.437963 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.438178 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.438273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260807 06:37:14.439754 31409 catalog_manager.cc:1347] Generated new cluster ID: 6ab2f4c8915a443ebf94ad9b13ba0b96
I20260807 06:37:14.439811 31409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:14.444378 31409 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:14.444895 31409 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:14.464830 31409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0: Generated new TSK 0
I20260807 06:37:14.465070 31409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:14.467214 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42651
--builtin_ntp_servers=127.2.243.20:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:14.550406 31422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:14.550696 31422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:14.550736 31422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:14.551978 31422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:14.552055 31422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:14.552103 31422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:37:14.553725 31422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:14.554018 31422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:14.554338 31422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:14.560474 31434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.560493 31432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.560483 31431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:14.560981 31422 server_base.cc:1034] running on GCE node
I20260807 06:37:14.561229 31422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:14.561458 31422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:14.562619 31422 hybrid_clock.cc:648] HybridClock initialized: now 1786084634562605 us; error 35 us; skew 500 ppm
I20260807 06:37:14.563853 31422 webserver.cc:460] Webserver started at http://127.2.243.1:36949/ using document root <none> and password file <none>
I20260807 06:37:14.564107 31422 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:14.564178 31422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:14.564319 31422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:14.565521 31422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "756df2acf096490dbc0ae4b632b1d21c"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "52de4b076905c8c5fc31ad53ad9ebdfe"
server_key_iv: "43330cf424b7c3e85ba4d7b77c09401e"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.565893 31422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "756df2acf096490dbc0ae4b632b1d21c"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "52de4b076905c8c5fc31ad53ad9ebdfe"
server_key_iv: "43330cf424b7c3e85ba4d7b77c09401e"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.567270 31422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:37:14.567978 31441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.568205 31422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:14.568342 31422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "756df2acf096490dbc0ae4b632b1d21c"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "52de4b076905c8c5fc31ad53ad9ebdfe"
server_key_iv: "43330cf424b7c3e85ba4d7b77c09401e"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.568423 31422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:14.576807 31422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:14.577111 31422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:14.577271 31422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:14.577517 31422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:14.577859 31422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:14.577919 31422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.577966 31422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:14.578006 31422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.584173 31422 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:46119
I20260807 06:37:14.584241 31626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:46119 every 8 connection(s)
I20260807 06:37:14.584663 31422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:37:14.590032 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.585510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:39807 (local address 127.2.243.62:42651)
0807 06:37:14.585585 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.585587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.585848 (+ 261us) server_negotiation.cc:407] Connection header received
0807 06:37:14.586283 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.586284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.586298 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.586356 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:14.586951 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.587507 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.588628 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.588828 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.588889 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.589808 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:14.589810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:14.589810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:14.589815 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:14.589830 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:14.589834 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:14.589884 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.589940 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.589963 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:37:14.590224 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.585514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42651 (local address 127.2.243.1:39807)
0807 06:37:14.585711 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.585742 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.586204 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.586403 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.586405 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.586413 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:14.586884 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.586888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.587603 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.587604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.588438 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.588440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.588914 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.588916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.589034 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.589655 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:14.589656 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:14.589684 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:14.589708 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:14.589885 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:14.589887 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:14.589889 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.589935 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.590015 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260807 06:37:14.590795 31627 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42651
I20260807 06:37:14.590910 31627 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:14.591110 31627 heartbeater.cc:511] Master 127.2.243.62:42651 requested a full tablet report, sending...
I20260807 06:37:14.591794 31333 ts_manager.cc:194] Registered new tserver with Master: 756df2acf096490dbc0ae4b632b1d21c (127.2.243.1:46119)
I20260807 06:37:14.592469 31333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.1:39807
I20260807 06:37:14.592941 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 31422
I20260807 06:37:14.593052 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:37:14.593271 3020 external_mini_cluster.cc:1428] Setting key 78f4612d432fe2efd61b877987b497d4
I20260807 06:37:14.594143 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42651
--builtin_ntp_servers=127.2.243.20:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:14.676244 31633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:14.676461 31633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:14.676496 31633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:14.677637 31633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:14.677701 31633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:14.677744 31633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:37:14.679057 31633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:14.679301 31633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:14.679574 31633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:14.684551 31641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.684563 31643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.684633 31640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:14.684921 31633 server_base.cc:1034] running on GCE node
I20260807 06:37:14.685149 31633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:14.685380 31633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:14.686537 31633 hybrid_clock.cc:648] HybridClock initialized: now 1786084634686514 us; error 52 us; skew 500 ppm
I20260807 06:37:14.687810 31633 webserver.cc:460] Webserver started at http://127.2.243.2:43323/ using document root <none> and password file <none>
I20260807 06:37:14.688071 31633 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:14.688140 31633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:14.688303 31633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:14.689507 31633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "ef926480ccdc4a1fb9396df17b108371"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "1b7121c11f712cb69e582e15fe85729d"
server_key_iv: "8e351600bf981ceb0e0c46b7fbb1bf0e"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.689872 31633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "ef926480ccdc4a1fb9396df17b108371"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "1b7121c11f712cb69e582e15fe85729d"
server_key_iv: "8e351600bf981ceb0e0c46b7fbb1bf0e"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.691169 31633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:14.691884 31653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.692049 31633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:14.692143 31633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "ef926480ccdc4a1fb9396df17b108371"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "1b7121c11f712cb69e582e15fe85729d"
server_key_iv: "8e351600bf981ceb0e0c46b7fbb1bf0e"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.692217 31633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:14.718204 31633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:14.718578 31633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:14.718745 31633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:14.719014 31633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:14.719378 31633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:14.719439 31633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.719477 31633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:14.719530 31633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.725765 31633 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:37367
I20260807 06:37:14.725832 31840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:37367 every 8 connection(s)
I20260807 06:37:14.726209 31633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:37:14.729828 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 31633
I20260807 06:37:14.729961 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:37:14.730161 3020 external_mini_cluster.cc:1428] Setting key 315b0beb355b069cb472043fd4af58b7
I20260807 06:37:14.731269 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42651
--builtin_ntp_servers=127.2.243.20:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:37:14.732307 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.727046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:45095 (local address 127.2.243.62:42651)
0807 06:37:14.727119 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.727122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.727381 (+ 259us) server_negotiation.cc:407] Connection header received
0807 06:37:14.727929 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.727930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.727941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.728005 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:14.728782 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.729279 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.730757 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.731004 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.731092 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.731972 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:14.731974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:14.731974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:14.731980 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:14.731999 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:14.732004 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:14.732072 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.732138 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.732166 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:37:14.732396 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.727007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42651 (local address 127.2.243.2:45095)
0807 06:37:14.727247 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.727295 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.727839 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.728066 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.728069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.728078 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:14.728700 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.728706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.729377 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.729378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.730472 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.730475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.731099 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.731100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.731217 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.731833 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:14.731833 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:14.731862 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:14.731883 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:14.732079 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:14.732084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:14.732086 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.732138 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.732214 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":125,"threads_started":1}
I20260807 06:37:14.732978 31841 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42651
I20260807 06:37:14.733105 31841 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:14.733302 31841 heartbeater.cc:511] Master 127.2.243.62:42651 requested a full tablet report, sending...
I20260807 06:37:14.733669 31333 ts_manager.cc:194] Registered new tserver with Master: ef926480ccdc4a1fb9396df17b108371 (127.2.243.2:37367)
I20260807 06:37:14.734055 31333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.2:45095
W20260807 06:37:14.815624 31846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:14.815819 31846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:14.815840 31846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:14.816900 31846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:14.816933 31846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:14.816957 31846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:37:14.818213 31846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:42651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:14.818387 31846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:14.818617 31846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:14.823493 31858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.823510 31855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:14.823514 31854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:14.823808 31846 server_base.cc:1034] running on GCE node
I20260807 06:37:14.824002 31846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:14.824205 31846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:14.825392 31846 hybrid_clock.cc:648] HybridClock initialized: now 1786084634825385 us; error 36 us; skew 500 ppm
I20260807 06:37:14.826737 31846 webserver.cc:460] Webserver started at http://127.2.243.3:40401/ using document root <none> and password file <none>
I20260807 06:37:14.827025 31846 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:14.827076 31846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:14.827214 31846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:14.828281 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "d266dbde9a7d4da991349fa84856c098"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "7376aad0a289fd543a8e6b4ddf0ec284"
server_key_iv: "48d07e963a58a129b923e61ff63cb2b7"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.828665 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "d266dbde9a7d4da991349fa84856c098"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "7376aad0a289fd543a8e6b4ddf0ec284"
server_key_iv: "48d07e963a58a129b923e61ff63cb2b7"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.829962 31846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:14.830670 31867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.830848 31846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:37:14.830941 31846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "d266dbde9a7d4da991349fa84856c098"
format_stamp: "Formatted at 2026-08-07 06:37:14 on dist-test-slave-rwrg"
server_key: "7376aad0a289fd543a8e6b4ddf0ec284"
server_key_iv: "48d07e963a58a129b923e61ff63cb2b7"
server_key_version: "encryptionkey@0"
I20260807 06:37:14.831033 31846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:14.849062 31846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:14.849377 31846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:14.849530 31846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:14.849766 31846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:14.850121 31846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:14.850172 31846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.850207 31846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:14.850227 31846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:14.855789 31846 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:36855
I20260807 06:37:14.855892 32048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:36855 every 8 connection(s)
I20260807 06:37:14.856195 31846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:37:14.857482 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 31846
I20260807 06:37:14.857617 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:37:14.857810 3020 external_mini_cluster.cc:1428] Setting key 595c80fa88a3d77e10a44167f524e8ae
I20260807 06:37:14.861864 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.857011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:58247 (local address 127.2.243.62:42651)
0807 06:37:14.857248 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.857250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.857469 (+ 219us) server_negotiation.cc:407] Connection header received
0807 06:37:14.858200 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.858201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.858213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.858286 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:14.859096 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.859597 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.860594 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.860785 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.860834 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.861631 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:14.861633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:14.861633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:14.861637 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:14.861651 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:14.861655 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:14.861699 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.861788 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.861808 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260807 06:37:14.862104 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.856977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:42651 (local address 127.2.243.3:58247)
0807 06:37:14.857335 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.857375 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.858072 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.858355 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.858359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.858371 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:14.859015 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.859021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.859699 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.859700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.860509 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.860512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.860862 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.860863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.860970 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.861456 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:14.861457 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:14.861523 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:14.861547 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:14.861715 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:14.861719 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:14.861722 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.861779 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.861857 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
I20260807 06:37:14.862686 32049 heartbeater.cc:347] Connected to a master server at 127.2.243.62:42651
I20260807 06:37:14.862800 32049 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:14.863029 32049 heartbeater.cc:511] Master 127.2.243.62:42651 requested a full tablet report, sending...
I20260807 06:37:14.863435 31333 ts_manager.cc:194] Registered new tserver with Master: d266dbde9a7d4da991349fa84856c098 (127.2.243.3:36855)
I20260807 06:37:14.863857 31333 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.3:58247
I20260807 06:37:14.869760 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:37:14.874394 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.871297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.2.243.62:42651)
0807 06:37:14.871371 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.871374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.871498 (+ 124us) server_negotiation.cc:407] Connection header received
0807 06:37:14.871573 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.871574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.871585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.871656 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:14.871859 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.872328 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.873166 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.873351 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.873427 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.874144 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:14.874146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:14.874146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:14.874150 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:14.874164 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:14.874174 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:14.874237 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.874291 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.874312 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 06:37:14.876251 31333 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:37:14.877954 31333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:37:14.884753 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.880445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.2.243.2:37367)
0807 06:37:14.881140 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.881145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.881155 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:14.881197 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.881199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.881215 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.881288 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:14.881384 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.882477 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.883789 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.884276 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.884384 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.884484 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.884606 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.884646 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":123,"threads_started":1}
I20260807 06:37:14.884950 32079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.880881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:46119 (local address 127.0.0.1:50184)
0807 06:37:14.881066 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.881077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.881133 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.881421 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.881422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.881424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:14.881461 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.881463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.882727 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.882728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.883718 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.883721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.884730 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.884731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.884790 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.884794 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.884845 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.884890 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260807 06:37:14.885051 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.880440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50672 (local address 127.2.243.3:36855)
0807 06:37:14.880696 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.880701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.880714 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:37:14.880755 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.880757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.880773 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.880868 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:14.881327 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.882119 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.883562 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.884256 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.884325 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.884412 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.884923 (+ 511us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.884961 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":152,"threads_started":1}
I20260807 06:37:14.884783 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.880320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:37367 (local address 127.0.0.1:53846)
0807 06:37:14.880676 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.880688 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.880749 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.881287 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.881287 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.881289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:14.881329 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.881330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.882575 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.882576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.883695 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.883698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.884388 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.884389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.884469 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.884483 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.884588 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.884644 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":81,"spinlock_wait_cycles":384,"thread_start_us":111,"threads_started":1}
I20260807 06:37:14.884776 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.880320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:36855 (local address 127.0.0.1:50672)
0807 06:37:14.880494 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:14.880587 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:14.880651 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:14.880930 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:14.880933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:14.880935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:14.881010 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.881013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.882226 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.882227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.883424 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:14.883427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.884450 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.884451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:14.884557 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.884562 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:37:14.884612 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:37:14.884658 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":70,"threads_started":1}
I20260807 06:37:14.885474 32077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:14.880648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.2.243.1:46119)
0807 06:37:14.880852 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:14.880857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:14.881205 (+ 348us) server_negotiation.cc:407] Connection header received
0807 06:37:14.881255 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:14.881257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:14.881276 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:14.881351 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:14.881575 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.882641 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.883816 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:14.884547 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:14.884614 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:14.884694 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:14.885151 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 06:37:14.885197 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":133,"threads_started":1}
I20260807 06:37:14.885932 31739 tablet_service.cc:1467] Processing CreateTablet for tablet 3b63c36a83204a5d9cfca4fa4ab8d243 (DEFAULT_TABLE table=test-table [id=54de117df2bd4d3ca4fe427645dbc611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:14.885960 31943 tablet_service.cc:1467] Processing CreateTablet for tablet 3b63c36a83204a5d9cfca4fa4ab8d243 (DEFAULT_TABLE table=test-table [id=54de117df2bd4d3ca4fe427645dbc611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:14.886260 31739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b63c36a83204a5d9cfca4fa4ab8d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:14.886348 31943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b63c36a83204a5d9cfca4fa4ab8d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:14.886301 31516 tablet_service.cc:1467] Processing CreateTablet for tablet 3b63c36a83204a5d9cfca4fa4ab8d243 (DEFAULT_TABLE table=test-table [id=54de117df2bd4d3ca4fe427645dbc611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:14.886670 31516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b63c36a83204a5d9cfca4fa4ab8d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:14.889082 32080 tablet_bootstrap.cc:492] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Bootstrap starting.
I20260807 06:37:14.889647 32080 tablet_bootstrap.cc:654] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:14.889951 32081 tablet_bootstrap.cc:492] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Bootstrap starting.
I20260807 06:37:14.889986 32080 log.cc:824] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:14.890377 32083 tablet_bootstrap.cc:492] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c: Bootstrap starting.
I20260807 06:37:14.890607 32081 tablet_bootstrap.cc:654] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:14.890885 32081 log.cc:824] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:14.891072 32083 tablet_bootstrap.cc:654] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:14.891434 32083 log.cc:824] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:14.891685 32081 tablet_bootstrap.cc:492] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: No bootstrap required, opened a new log
I20260807 06:37:14.891775 32081 ts_tablet_manager.cc:1397] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:37:14.891793 32080 tablet_bootstrap.cc:492] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: No bootstrap required, opened a new log
I20260807 06:37:14.891891 32080 ts_tablet_manager.cc:1397] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:37:14.892163 32083 tablet_bootstrap.cc:492] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c: No bootstrap required, opened a new log
I20260807 06:37:14.892251 32083 ts_tablet_manager.cc:1397] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:37:14.894146 32083 raft_consensus.cc:348] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:14.894343 32083 raft_consensus.cc:374] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:14.894397 32083 raft_consensus.cc:729] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756df2acf096490dbc0ae4b632b1d21c, State: Initialized, Role: FOLLOWER
I20260807 06:37:14.894460 32080 raft_consensus.cc:348] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:14.894517 32083 consensus_queue.cc:260] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:14.894380 32081 raft_consensus.cc:348] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:14.894610 32080 raft_consensus.cc:374] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:14.894644 32081 raft_consensus.cc:374] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:14.894649 32080 raft_consensus.cc:729] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef926480ccdc4a1fb9396df17b108371, State: Initialized, Role: FOLLOWER
I20260807 06:37:14.894690 32081 raft_consensus.cc:729] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d266dbde9a7d4da991349fa84856c098, State: Initialized, Role: FOLLOWER
I20260807 06:37:14.894829 32083 ts_tablet_manager.cc:1428] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:37:14.894805 32080 consensus_queue.cc:260] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:14.894846 32081 consensus_queue.cc:260] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:14.895001 31627 heartbeater.cc:503] Master 127.2.243.62:42651 was elected leader, sending a full tablet report...
I20260807 06:37:14.895066 32081 ts_tablet_manager.cc:1428] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 06:37:14.895067 32080 ts_tablet_manager.cc:1428] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 06:37:14.895183 32049 heartbeater.cc:503] Master 127.2.243.62:42651 was elected leader, sending a full tablet report...
I20260807 06:37:14.895452 31841 heartbeater.cc:503] Master 127.2.243.62:42651 was elected leader, sending a full tablet report...
W20260807 06:37:14.976823 31843 tablet.cc:2369] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:37:15.085438 31629 tablet.cc:2369] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:37:15.106817 32052 tablet.cc:2369] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:37:15.119755 32092 raft_consensus.cc:482] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:15.119938 32092 raft_consensus.cc:504] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:15.120352 32092 leader_election.cc:288] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef926480ccdc4a1fb9396df17b108371 (127.2.243.2:37367), 756df2acf096490dbc0ae4b632b1d21c (127.2.243.1:46119)
I20260807 06:37:15.124639 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:15.120677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:37367 (local address 127.2.243.3:40061)
0807 06:37:15.120759 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:15.120774 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:15.120833 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:15.121203 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:15.121204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:15.121208 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:15.121307 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:15.121309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.122258 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.122259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:15.123544 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:15.123547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.124317 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.124318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:15.124409 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:15.124414 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:37:15.124486 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:37:15.124553 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":55}
I20260807 06:37:15.124661 32077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:15.120943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:48093 (local address 127.2.243.1:46119)
0807 06:37:15.121005 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:15.121007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:15.121147 (+ 140us) server_negotiation.cc:407] Connection header received
0807 06:37:15.121175 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:15.121176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:15.121186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:15.121237 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:15.121399 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.122155 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.123655 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.124200 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.124283 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:15.124293 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:15.124562 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 06:37:15.124602 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:37:15.124658 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:15.120928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:40061 (local address 127.2.243.2:37367)
0807 06:37:15.120988 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:15.120991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:15.121002 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:37:15.121043 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:15.121044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:15.121054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:15.121173 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:15.121399 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.122167 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.123654 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.124200 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.124283 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:15.124293 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:15.124533 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 06:37:15.124589 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:37:15.124648 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:15.120828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:46119 (local address 127.2.243.3:48093)
0807 06:37:15.121025 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:15.121034 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:15.121097 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:15.121256 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:15.121256 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:15.121258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:15.121316 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:15.121318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.122258 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.122259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:15.123565 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:15.123567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.124318 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.124319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:15.124411 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:15.124416 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:37:15.124486 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:37:15.124581 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":38,"thread_start_us":112,"threads_started":1}
I20260807 06:37:15.125041 31554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b63c36a83204a5d9cfca4fa4ab8d243" candidate_uuid: "d266dbde9a7d4da991349fa84856c098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756df2acf096490dbc0ae4b632b1d21c" is_pre_election: true
I20260807 06:37:15.125088 31772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b63c36a83204a5d9cfca4fa4ab8d243" candidate_uuid: "d266dbde9a7d4da991349fa84856c098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef926480ccdc4a1fb9396df17b108371" is_pre_election: true
I20260807 06:37:15.125178 31554 raft_consensus.cc:2454] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d266dbde9a7d4da991349fa84856c098 in term 0.
I20260807 06:37:15.125264 31772 raft_consensus.cc:2454] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d266dbde9a7d4da991349fa84856c098 in term 0.
I20260807 06:37:15.125452 31868 leader_election.cc:302] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756df2acf096490dbc0ae4b632b1d21c, d266dbde9a7d4da991349fa84856c098; no voters:
I20260807 06:37:15.125581 32092 raft_consensus.cc:2781] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:37:15.125638 32092 raft_consensus.cc:482] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:37:15.125662 32092 raft_consensus.cc:3037] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:15.126344 32092 raft_consensus.cc:504] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:15.126474 32092 leader_election.cc:288] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [CANDIDATE]: Term 1 election: Requested vote from peers ef926480ccdc4a1fb9396df17b108371 (127.2.243.2:37367), 756df2acf096490dbc0ae4b632b1d21c (127.2.243.1:46119)
I20260807 06:37:15.126662 31772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b63c36a83204a5d9cfca4fa4ab8d243" candidate_uuid: "d266dbde9a7d4da991349fa84856c098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef926480ccdc4a1fb9396df17b108371"
I20260807 06:37:15.126681 31554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b63c36a83204a5d9cfca4fa4ab8d243" candidate_uuid: "d266dbde9a7d4da991349fa84856c098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756df2acf096490dbc0ae4b632b1d21c"
I20260807 06:37:15.126749 31554 raft_consensus.cc:3037] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:15.126775 31772 raft_consensus.cc:3037] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:15.127444 31554 raft_consensus.cc:2454] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d266dbde9a7d4da991349fa84856c098 in term 1.
I20260807 06:37:15.127620 31868 leader_election.cc:302] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756df2acf096490dbc0ae4b632b1d21c, d266dbde9a7d4da991349fa84856c098; no voters:
I20260807 06:37:15.127722 32092 raft_consensus.cc:2781] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:15.127722 31772 raft_consensus.cc:2454] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d266dbde9a7d4da991349fa84856c098 in term 1.
I20260807 06:37:15.127889 32092 raft_consensus.cc:686] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 1 LEADER]: Becoming Leader. State: Replica: d266dbde9a7d4da991349fa84856c098, State: Running, Role: LEADER
I20260807 06:37:15.128022 32092 consensus_queue.cc:237] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } }
I20260807 06:37:15.129011 31333 catalog_manager.cc:5495] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 reported cstate change: term changed from 0 to 1, leader changed from <none> to d266dbde9a7d4da991349fa84856c098 (127.2.243.3). New cstate: current_term: 1 leader_uuid: "d266dbde9a7d4da991349fa84856c098" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d266dbde9a7d4da991349fa84856c098" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 36855 } health_report { overall_health: HEALTHY } } }
I20260807 06:37:15.166990 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:15.164533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.2.243.3:36855)
0807 06:37:15.164625 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:15.164632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:15.164643 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:37:15.164715 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:15.164716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:15.164736 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:15.164806 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:15.164933 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.165390 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.166135 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:15.166312 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:15.166363 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:15.166503 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:15.166605 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:15.166667 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:15.166804 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 06:37:15.166855 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:37:15.169097 31772 raft_consensus.cc:1261] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 1 FOLLOWER]: Refusing update from remote peer d266dbde9a7d4da991349fa84856c098: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:15.169097 31554 raft_consensus.cc:1261] T 3b63c36a83204a5d9cfca4fa4ab8d243 P 756df2acf096490dbc0ae4b632b1d21c [term 1 FOLLOWER]: Refusing update from remote peer d266dbde9a7d4da991349fa84856c098: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:15.169601 32092 consensus_queue.cc:1035] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef926480ccdc4a1fb9396df17b108371" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:15.169775 32094 consensus_queue.cc:1035] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [LEADER]: Connected to new peer: Peer: permanent_uuid: "756df2acf096490dbc0ae4b632b1d21c" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:15.172611 32098 mvcc.cc:205] Tried to move back new op lower bound from 7315802665649356800 to 7315802665485111296. Current Snapshot: MvccSnapshot[applied={T|T < 7315802665649356800}]
I20260807 06:37:15.172828 32100 mvcc.cc:205] Tried to move back new op lower bound from 7315802665649356800 to 7315802665485111296. Current Snapshot: MvccSnapshot[applied={T|T < 7315802665649356800}]
I20260807 06:37:15.173071 32099 mvcc.cc:205] Tried to move back new op lower bound from 7315802665649356800 to 7315802665485111296. Current Snapshot: MvccSnapshot[applied={T|T < 7315802665649356800}]
I20260807 06:37:15.175048 31332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:15.175159 31332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:15.175983 31332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0cad99e49814cb4b53e679e29c028e0: Sending DeleteTablet for 3 replicas of tablet 3b63c36a83204a5d9cfca4fa4ab8d243
I20260807 06:37:15.176347 31943 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b63c36a83204a5d9cfca4fa4ab8d243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:15 UTC) from {username='slave'} at 127.0.0.1:50672
I20260807 06:37:15.176451 31739 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b63c36a83204a5d9cfca4fa4ab8d243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:15 UTC) from {username='slave'} at 127.0.0.1:53846
I20260807 06:37:15.176765 32118 tablet_replica.cc:330] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: stopping tablet replica
I20260807 06:37:15.176796 31516 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b63c36a83204a5d9cfca4fa4ab8d243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:15 UTC) from {username='slave'} at 127.0.0.1:50184
I20260807 06:37:15.176923 32118 raft_consensus.cc:2229] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:15.176959 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 31422
I20260807 06:37:15.177124 32118 raft_consensus.cc:2258] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:15.177577 32117 tablet_replica.cc:330] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: stopping tablet replica
I20260807 06:37:15.177846 32117 raft_consensus.cc:2229] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:37:15.177969 32117 raft_consensus.cc:2258] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:15.180188 32117 ts_tablet_manager.cc:1905] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:15.181898 32117 ts_tablet_manager.cc:1918] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:15.181963 32117 log.cc:1196] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/3b63c36a83204a5d9cfca4fa4ab8d243
I20260807 06:37:15.182233 32117 ts_tablet_manager.cc:1939] T 3b63c36a83204a5d9cfca4fa4ab8d243 P d266dbde9a7d4da991349fa84856c098: Deleting consensus metadata
I20260807 06:37:15.183552 31308 catalog_manager.cc:4841] TS d266dbde9a7d4da991349fa84856c098 (127.2.243.3:36855): tablet 3b63c36a83204a5d9cfca4fa4ab8d243 (table test-table [id=54de117df2bd4d3ca4fe427645dbc611]) successfully deleted
I20260807 06:37:15.192927 32118 ts_tablet_manager.cc:1905] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:15.194486 32118 ts_tablet_manager.cc:1918] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:15.194588 32118 log.cc:1196] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/3b63c36a83204a5d9cfca4fa4ab8d243
I20260807 06:37:15.194837 32118 ts_tablet_manager.cc:1939] T 3b63c36a83204a5d9cfca4fa4ab8d243 P ef926480ccdc4a1fb9396df17b108371: Deleting consensus metadata
W20260807 06:37:15.195295 31304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:46119 (error 108)
I20260807 06:37:15.195583 31304 catalog_manager.cc:4841] TS ef926480ccdc4a1fb9396df17b108371 (127.2.243.2:37367): tablet 3b63c36a83204a5d9cfca4fa4ab8d243 (table test-table [id=54de117df2bd4d3ca4fe427645dbc611]) successfully deleted
W20260807 06:37:15.195891 32068 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:15.195764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:46119 (local address 127.0.0.1:50196)
0807 06:37:15.195817 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:15.195857 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:46119: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 06:37:15.196056 31304 catalog_manager.cc:4584] TS 756df2acf096490dbc0ae4b632b1d21c (127.2.243.1:46119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b63c36a83204a5d9cfca4fa4ab8d243: Network error: Client connection negotiation failed: client connection to 127.2.243.1:46119: connect: Connection reset by peer (error 104)
I20260807 06:37:15.196259 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 31633
I20260807 06:37:15.206727 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 31846
I20260807 06:37:15.217867 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 31277
2026-08-07T06:37:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (955 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:15 dist-test-slave-rwrg krb5kdc[32130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:15 dist-test-slave-rwrg krb5kdc[32130](info): set up 2 sockets
Aug 07 06:37:15 dist-test-slave-rwrg krb5kdc[32130](info): commencing operation
krb5kdc: starting...
W20260807 06:37:16.470726 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:16 dist-test-slave-rwrg krb5kdc[32130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:16.595026 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43035
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:43035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:16.679235 32157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:16.679441 32157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:16.679464 32157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:16.680718 32157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:16.680763 32157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:16.680778 32157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:16.680791 32157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:16.680845 32157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:16.682350 32157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:43035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:16.682614 32157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:16.682861 32157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:16.687825 32169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:16.687813 32164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:16.687882 32166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:16.688028 32157 server_base.cc:1034] running on GCE node
I20260807 06:37:16.688390 32157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:16.688721 32157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:16.689898 32157 hybrid_clock.cc:648] HybridClock initialized: now 1786084636689884 us; error 30 us; skew 500 ppm
Aug 07 06:37:16 dist-test-slave-rwrg krb5kdc[32130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:16.692795 32157 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:16.693313 32157 webserver.cc:460] Webserver started at http://127.2.243.62:38509/ using document root <none> and password file <none>
I20260807 06:37:16.693584 32157 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:16.693660 32157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:16.693835 32157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:16.694897 32157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "7db8210bcdb946669d7c8ece4a071626"
format_stamp: "Formatted at 2026-08-07 06:37:16 on dist-test-slave-rwrg"
server_key: "0761703ee32fa9e7a492b66f3c57192f"
server_key_iv: "393409b13f6e5927cbc0b9cb6ea8343c"
server_key_version: "encryptionkey@0"
I20260807 06:37:16.695281 32157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "7db8210bcdb946669d7c8ece4a071626"
format_stamp: "Formatted at 2026-08-07 06:37:16 on dist-test-slave-rwrg"
server_key: "0761703ee32fa9e7a492b66f3c57192f"
server_key_iv: "393409b13f6e5927cbc0b9cb6ea8343c"
server_key_version: "encryptionkey@0"
I20260807 06:37:16.696820 32157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:16.697578 32181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:16.697753 32157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:16.697863 32157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "7db8210bcdb946669d7c8ece4a071626"
format_stamp: "Formatted at 2026-08-07 06:37:16 on dist-test-slave-rwrg"
server_key: "0761703ee32fa9e7a492b66f3c57192f"
server_key_iv: "393409b13f6e5927cbc0b9cb6ea8343c"
server_key_version: "encryptionkey@0"
I20260807 06:37:16.697939 32157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:16.712918 32157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:16.715215 32157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:16.715381 32157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:16.719496 32157 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:43035
I20260807 06:37:16.719530 32266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:43035 every 8 connection(s)
I20260807 06:37:16.719961 32157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:16.720485 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 32157
I20260807 06:37:16.720677 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:16.720892 3020 external_mini_cluster.cc:1428] Setting key 2d4b5a14c90583cd8eb89c45167d3305
I20260807 06:37:16.721109 32267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:16.723836 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626: Bootstrap starting.
I20260807 06:37:16.724460 32267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:16.724830 32267 log.cc:824] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:16.725557 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626: No bootstrap required, opened a new log
Aug 07 06:37:16 dist-test-slave-rwrg krb5kdc[32130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:16.727423 32267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db8210bcdb946669d7c8ece4a071626" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43035 } }
I20260807 06:37:16.727582 32267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:16.727627 32267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db8210bcdb946669d7c8ece4a071626, State: Initialized, Role: FOLLOWER
I20260807 06:37:16.727763 32267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db8210bcdb946669d7c8ece4a071626" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43035 } }
I20260807 06:37:16.727854 32267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:16.727892 32267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:16.727948 32267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:16.728739 32267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db8210bcdb946669d7c8ece4a071626" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43035 } }
I20260807 06:37:16.728864 32267 leader_election.cc:302] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7db8210bcdb946669d7c8ece4a071626; no voters:
I20260807 06:37:16.729027 32267 leader_election.cc:288] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:16.729102 32274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:16.729300 32274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [term 1 LEADER]: Becoming Leader. State: Replica: 7db8210bcdb946669d7c8ece4a071626, State: Running, Role: LEADER
I20260807 06:37:16.729323 32267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:16.729453 32274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db8210bcdb946669d7c8ece4a071626" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43035 } }
I20260807 06:37:16.729827 32277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7db8210bcdb946669d7c8ece4a071626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db8210bcdb946669d7c8ece4a071626" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43035 } } }
I20260807 06:37:16.729910 32277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:16.730146 32288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:16.730849 32288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:16.730981 32276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7db8210bcdb946669d7c8ece4a071626. Latest consensus state: current_term: 1 leader_uuid: "7db8210bcdb946669d7c8ece4a071626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db8210bcdb946669d7c8ece4a071626" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43035 } } }
I20260807 06:37:16.731206 32276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:16.731878 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:16.721417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.2.243.62:43035)
0807 06:37:16.721632 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:16.721640 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:16.721665 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:37:16.722175 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:16.722181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:16.722227 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:16.722374 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:16.723017 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:16.723832 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:16.724661 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:16.725194 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:16.725339 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:16.727630 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:16.727634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:16.727634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:16.727647 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:16.729705 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:16.730147 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:16.730150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:16.730150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:16.730313 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:16.730587 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:16.730588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:16.730588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:16.730915 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:16.731068 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:16.731540 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 06:37:16.731689 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260807 06:37:16.732571 32288 catalog_manager.cc:1347] Generated new cluster ID: 5b2173ebb4fb45108c084f82c7b769bc
I20260807 06:37:16.732645 32288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:16.745200 32288 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:16.745738 32288 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:16.752869 32288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626: Generated new TSK 0
I20260807 06:37:16.753078 32288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:37:16.783787 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43035
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:16.866189 32308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:16.866390 32308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:16.866413 32308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:16.867565 32308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:16.867610 32308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:16.867636 32308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:37:16.869175 32308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:16.869433 32308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:16.869675 32308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:16.874778 32322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:16.874784 32317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:16.874851 32319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:16.875149 32308 server_base.cc:1034] running on GCE node
I20260807 06:37:16.875337 32308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:16.875586 32308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:16.876735 32308 hybrid_clock.cc:648] HybridClock initialized: now 1786084636876690 us; error 64 us; skew 500 ppm
Aug 07 06:37:16 dist-test-slave-rwrg krb5kdc[32130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:16.882877 32308 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:37:16.883332 32308 webserver.cc:460] Webserver started at http://127.2.243.1:41737/ using document root <none> and password file <none>
I20260807 06:37:16.883569 32308 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:16.883615 32308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:16.883747 32308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:16.884907 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "a30e5602c6af4b33ba899b24d2c94048"
format_stamp: "Formatted at 2026-08-07 06:37:16 on dist-test-slave-rwrg"
server_key: "b3af801355202d4c2bd6c7e6d0535f23"
server_key_iv: "bad935a167d3a017dd9036594b8c9aad"
server_key_version: "encryptionkey@0"
I20260807 06:37:16.885289 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "a30e5602c6af4b33ba899b24d2c94048"
format_stamp: "Formatted at 2026-08-07 06:37:16 on dist-test-slave-rwrg"
server_key: "b3af801355202d4c2bd6c7e6d0535f23"
server_key_iv: "bad935a167d3a017dd9036594b8c9aad"
server_key_version: "encryptionkey@0"
I20260807 06:37:16.886683 32308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:16.887523 32330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:16.887744 32308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:16.887859 32308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "a30e5602c6af4b33ba899b24d2c94048"
format_stamp: "Formatted at 2026-08-07 06:37:16 on dist-test-slave-rwrg"
server_key: "b3af801355202d4c2bd6c7e6d0535f23"
server_key_iv: "bad935a167d3a017dd9036594b8c9aad"
server_key_version: "encryptionkey@0"
I20260807 06:37:16.887935 32308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:16.896051 32308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:16.898286 32308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:16.898456 32308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:16.898686 32308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:16.898995 32308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:16.899057 32308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:16.899091 32308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:16.899120 32308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:16.905290 32308 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:40529
I20260807 06:37:16.905341 32535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:40529 every 8 connection(s)
I20260807 06:37:16.905635 32308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:37:16.909463 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 32308
I20260807 06:37:16.909649 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:37:16.909878 3020 external_mini_cluster.cc:1428] Setting key 9985aa397f0a076601fcedccfa797509
Aug 07 06:37:16 dist-test-slave-rwrg krb5kdc[32130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:16.915362 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:16.906327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:53195 (local address 127.2.243.62:43035)
0807 06:37:16.906399 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:16.906401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:16.906583 (+ 182us) server_negotiation.cc:407] Connection header received
0807 06:37:16.907071 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:16.907072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:16.907083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:16.907136 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:16.908041 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:16.908569 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:16.909611 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:16.909889 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:16.909963 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:16.912426 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:16.912427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:16.912428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:16.912436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:16.913920 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:16.914341 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:16.914342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:16.914343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:16.914406 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:16.914711 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:16.914712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:16.914712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:16.914827 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:16.914896 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:16.915186 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 06:37:16.915271 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:37:16.915539 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:16.906316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43035 (local address 127.2.243.1:53195)
0807 06:37:16.906479 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:16.906516 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:16.906974 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:16.907208 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:16.907211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:16.907762 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:16.907948 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:16.907952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:16.908653 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:16.908654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:16.909506 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:16.909509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:16.909971 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:16.909973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:16.910125 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:16.910677 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:16.910686 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:16.912326 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:16.914033 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:16.914040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:16.914042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:16.914261 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:16.914516 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:16.914517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:16.914517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:16.914634 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:16.914900 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:16.914907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:16.915049 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 06:37:16.915153 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 06:37:16.915307 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260807 06:37:16.916069 32537 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43035
I20260807 06:37:16.916185 32537 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:16.916400 32537 heartbeater.cc:511] Master 127.2.243.62:43035 requested a full tablet report, sending...
I20260807 06:37:16.917107 32210 ts_manager.cc:194] Registered new tserver with Master: a30e5602c6af4b33ba899b24d2c94048 (127.2.243.1:40529)
I20260807 06:37:16.918072 32210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:53195
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:37:16.937880 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43035
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:17.019757 32544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:17.019945 32544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:17.019968 32544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:17.021142 32544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:17.021181 32544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:17.021207 32544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:37:17.022661 32544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:17.022893 32544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:17.023114 32544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:17.028227 32558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.028322 32553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.028472 32544 server_base.cc:1034] running on GCE node
W20260807 06:37:17.028590 32552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.028861 32544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:17.029171 32544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:17.030368 32544 hybrid_clock.cc:648] HybridClock initialized: now 1786084637030350 us; error 36 us; skew 500 ppm
Aug 07 06:37:17 dist-test-slave-rwrg krb5kdc[32130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084637, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:17.033030 32544 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:37:17.033515 32544 webserver.cc:460] Webserver started at http://127.2.243.2:34477/ using document root <none> and password file <none>
I20260807 06:37:17.033723 32544 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:17.033763 32544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:17.033852 32544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:17.034834 32544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "31fd21e2ef674aeb85d13f8758dbe696"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "7b342b82f76746811d0ae3af19ceaa5b"
server_key_iv: "c26d34ebfe92f9ed3b656290c22968bd"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.035144 32544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "31fd21e2ef674aeb85d13f8758dbe696"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "7b342b82f76746811d0ae3af19ceaa5b"
server_key_iv: "c26d34ebfe92f9ed3b656290c22968bd"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.036379 32544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:17.037184 32566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.037410 32544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:37:17.037503 32544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "31fd21e2ef674aeb85d13f8758dbe696"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "7b342b82f76746811d0ae3af19ceaa5b"
server_key_iv: "c26d34ebfe92f9ed3b656290c22968bd"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.037583 32544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:17.051342 32544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:17.053340 32544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:17.053504 32544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:17.053745 32544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:17.054062 32544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:17.054116 32544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.054159 32544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:17.054188 32544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.060235 32544 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:36331
I20260807 06:37:17.060286 32762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:36331 every 8 connection(s)
I20260807 06:37:17.060608 32544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:37:17.062825 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 32544
I20260807 06:37:17.062953 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:37:17.063197 3020 external_mini_cluster.cc:1428] Setting key 511e01a8dd4d6cab3720c98533e48071
Aug 07 06:37:17 dist-test-slave-rwrg krb5kdc[32130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084637, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:17.071118 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.061366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:36629 (local address 127.2.243.62:43035)
0807 06:37:17.061411 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.061413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.061607 (+ 194us) server_negotiation.cc:407] Connection header received
0807 06:37:17.062054 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.062054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.062065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.062136 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.063084 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.063706 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.064905 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.065106 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.065205 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.067514 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.067517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.067517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:17.067526 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.069570 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:17.070039 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:17.070040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:17.070041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:17.070113 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:17.070384 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:17.070385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:17.070385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:17.070549 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.070634 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.070896 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.070994 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:17.071244 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.061311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43035 (local address 127.2.243.2:36629)
0807 06:37:17.061473 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.061514 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.061973 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.062186 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.062189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.062740 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:17.062921 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.062950 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.063824 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.063825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.064792 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.064795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.065230 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.065231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.065369 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.065941 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:17.065949 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:17.067402 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:17.069680 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:17.069686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:17.069688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:17.069948 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:17.070197 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:17.070198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:17.070198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:17.070306 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:17.070627 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:17.070632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:17.070749 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.070838 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.070963 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260807 06:37:17.071796 32763 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43035
I20260807 06:37:17.071892 32763 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:17.072109 32763 heartbeater.cc:511] Master 127.2.243.62:43035 requested a full tablet report, sending...
I20260807 06:37:17.072649 32210 ts_manager.cc:194] Registered new tserver with Master: 31fd21e2ef674aeb85d13f8758dbe696 (127.2.243.2:36331)
I20260807 06:37:17.073068 32210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:36629
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:37:17.090739 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43035
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:17.173878 301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:17.174111 301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:17.174150 301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:17.175262 301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:17.175328 301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:17.175372 301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:37:17.176833 301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:43035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:17.177093 301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:17.177318 301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:17.182371 308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.182382 313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.182406 309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.182657 301 server_base.cc:1034] running on GCE node
I20260807 06:37:17.182858 301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:17.183096 301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:17.184294 301 hybrid_clock.cc:648] HybridClock initialized: now 1786084637184252 us; error 54 us; skew 500 ppm
Aug 07 06:37:17 dist-test-slave-rwrg krb5kdc[32130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084637, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:17.187114 301 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:37:17.187669 301 webserver.cc:460] Webserver started at http://127.2.243.3:33857/ using document root <none> and password file <none>
I20260807 06:37:17.187924 301 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:17.187973 301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:17.188159 301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:17.189368 301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "87768a4d39744cbf8460b4d15e7b245d"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "12021921653a9a383a1bb661840bdc35"
server_key_iv: "a9a65868739de7d6b515f6bc583a9b02"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.189754 301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "87768a4d39744cbf8460b4d15e7b245d"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "12021921653a9a383a1bb661840bdc35"
server_key_iv: "a9a65868739de7d6b515f6bc583a9b02"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.191354 301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:17.192150 327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.192368 301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:17.192513 301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "87768a4d39744cbf8460b4d15e7b245d"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "12021921653a9a383a1bb661840bdc35"
server_key_iv: "a9a65868739de7d6b515f6bc583a9b02"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.192615 301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:17.201686 301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:17.203814 301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:17.203969 301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:17.204193 301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:17.204603 301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:17.204653 301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.204676 301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:17.204715 301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.210397 301 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:38985
I20260807 06:37:17.210472 509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:38985 every 8 connection(s)
I20260807 06:37:17.210801 301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:37:17.216006 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 301
I20260807 06:37:17.216106 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:37:17.216318 3020 external_mini_cluster.cc:1428] Setting key 3828330b4f10b01210319c4bae21f61f
Aug 07 06:37:17 dist-test-slave-rwrg krb5kdc[32130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084637, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:17.219930 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.211535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:47609 (local address 127.2.243.62:43035)
0807 06:37:17.211606 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.211609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.211838 (+ 229us) server_negotiation.cc:407] Connection header received
0807 06:37:17.212284 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.212285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.212295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.212347 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.213233 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.213756 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.214714 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.214914 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.214971 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.217327 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.217329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.217329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:17.217338 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.218776 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:17.219125 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:17.219126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:17.219126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:17.219176 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:17.219380 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:17.219380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:17.219381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:17.219475 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.219523 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.219788 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.219869 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 06:37:17.220115 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.211508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43035 (local address 127.2.243.3:47609)
0807 06:37:17.211721 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.211756 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.212204 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.212398 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.212401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.212971 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:17.213155 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.213160 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.213840 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.213841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.214621 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.214623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.214992 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.214993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.215085 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.215625 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:17.215634 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:17.217246 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:17.218846 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:17.218851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:17.218853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:17.219061 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:17.219233 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:17.219234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:17.219234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:17.219327 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:17.219559 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:17.219563 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:17.219668 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.219762 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.219892 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":132,"threads_started":1}
I20260807 06:37:17.220686 510 heartbeater.cc:347] Connected to a master server at 127.2.243.62:43035
I20260807 06:37:17.220783 510 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:17.220952 510 heartbeater.cc:511] Master 127.2.243.62:43035 requested a full tablet report, sending...
I20260807 06:37:17.221289 32210 ts_manager.cc:194] Registered new tserver with Master: 87768a4d39744cbf8460b4d15e7b245d (127.2.243.3:38985)
I20260807 06:37:17.221650 32210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:47609
I20260807 06:37:17.228159 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:37:17.236294 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.229642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.2.243.62:43035)
0807 06:37:17.229704 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.229706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.229785 (+ 79us) server_negotiation.cc:407] Connection header received
0807 06:37:17.229861 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.229862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.229872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.229942 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.230633 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.231247 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.232130 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.232303 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.232356 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.233887 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.233888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.233888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:17.233895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.235163 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:17.235572 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:17.235573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:17.235573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:17.235628 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:17.235854 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:17.235855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:17.235855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:17.235985 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.236089 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.236181 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.236232 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:37:17.237989 32210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:37:17.239552 32210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:37:17.246552 528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.241784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38985 (local address 127.0.0.1:53778)
0807 06:37:17.242363 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.242374 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.242431 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.242669 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.242670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.242672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.242708 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.242709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.243923 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.243924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.245407 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.245410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.246179 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.246180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.246282 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.246299 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.246379 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.246431 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":73,"thread_start_us":55,"threads_started":1}
I20260807 06:37:17.246658 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.241755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40529 (local address 127.0.0.1:60362)
0807 06:37:17.242111 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.242122 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.242161 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.242448 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.242449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.242451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.242506 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.242510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.244865 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.244866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.245681 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.245684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.246404 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.246405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.246479 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.246483 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.246544 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.246579 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":3,"thread_start_us":58,"threads_started":1}
I20260807 06:37:17.246896 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.242179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.2.243.3:38985)
0807 06:37:17.242375 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.242379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.242422 (+ 43us) server_negotiation.cc:407] Connection header received
0807 06:37:17.242494 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.242496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.242519 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.242593 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.242774 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.243824 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.245495 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.246101 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.246156 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.246194 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.246780 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.246819 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260807 06:37:17.246908 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.241966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.2.243.1:40529)
0807 06:37:17.242194 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.242200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.242209 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:17.242247 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.242248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.242265 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.242369 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.243882 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.244750 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.245767 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.246311 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.246387 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.246440 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.246575 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.246620 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":117,"threads_started":1}
I20260807 06:37:17.247507 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.241755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:36331 (local address 127.0.0.1:39848)
0807 06:37:17.241911 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.241950 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.242013 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.242423 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.242428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.242430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.242506 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.242507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.243870 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.243871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.245409 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.245410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.247346 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.247347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.247392 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.247395 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.247436 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.247463 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":40,"thread_start_us":55,"threads_started":1}
I20260807 06:37:17.247597 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.241953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.2.243.2:36331)
0807 06:37:17.242157 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.242161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.242173 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:37:17.242223 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.242225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.242242 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.242327 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.242595 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.243740 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.246224 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.247267 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.247346 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.247397 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.247459 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.247498 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260807 06:37:17.247572 405 tablet_service.cc:1467] Processing CreateTablet for tablet 89b06476f9a747c39423cdc9c4c1f5fe (DEFAULT_TABLE table=test-table [id=b946e19c5db24caa873dbd9372f52150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:17.247948 405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b06476f9a747c39423cdc9c4c1f5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:17.248246 32647 tablet_service.cc:1467] Processing CreateTablet for tablet 89b06476f9a747c39423cdc9c4c1f5fe (DEFAULT_TABLE table=test-table [id=b946e19c5db24caa873dbd9372f52150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:17.248620 32647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b06476f9a747c39423cdc9c4c1f5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:17.248921 32424 tablet_service.cc:1467] Processing CreateTablet for tablet 89b06476f9a747c39423cdc9c4c1f5fe (DEFAULT_TABLE table=test-table [id=b946e19c5db24caa873dbd9372f52150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:17.249215 32424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b06476f9a747c39423cdc9c4c1f5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:17.251425 539 tablet_bootstrap.cc:492] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Bootstrap starting.
I20260807 06:37:17.251915 540 tablet_bootstrap.cc:492] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048: Bootstrap starting.
I20260807 06:37:17.252182 539 tablet_bootstrap.cc:654] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:17.252173 541 tablet_bootstrap.cc:492] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Bootstrap starting.
I20260807 06:37:17.252619 540 tablet_bootstrap.cc:654] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:17.252797 539 log.cc:824] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:17.252856 541 tablet_bootstrap.cc:654] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:17.253193 540 log.cc:824] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:17.253429 541 log.cc:824] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:17.253793 539 tablet_bootstrap.cc:492] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: No bootstrap required, opened a new log
I20260807 06:37:17.253870 539 ts_tablet_manager.cc:1397] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:37:17.254109 540 tablet_bootstrap.cc:492] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048: No bootstrap required, opened a new log
I20260807 06:37:17.254166 540 ts_tablet_manager.cc:1397] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:37:17.254181 541 tablet_bootstrap.cc:492] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: No bootstrap required, opened a new log
I20260807 06:37:17.254232 541 ts_tablet_manager.cc:1397] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:37:17.255128 539 raft_consensus.cc:348] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.255256 539 raft_consensus.cc:374] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:17.255277 539 raft_consensus.cc:729] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87768a4d39744cbf8460b4d15e7b245d, State: Initialized, Role: FOLLOWER
I20260807 06:37:17.255393 539 consensus_queue.cc:260] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.255667 539 ts_tablet_manager.cc:1428] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:37:17.255699 510 heartbeater.cc:503] Master 127.2.243.62:43035 was elected leader, sending a full tablet report...
I20260807 06:37:17.256037 540 raft_consensus.cc:348] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.256044 541 raft_consensus.cc:348] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.256209 540 raft_consensus.cc:374] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:17.256209 541 raft_consensus.cc:374] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:17.256245 541 raft_consensus.cc:729] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fd21e2ef674aeb85d13f8758dbe696, State: Initialized, Role: FOLLOWER
I20260807 06:37:17.256245 540 raft_consensus.cc:729] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30e5602c6af4b33ba899b24d2c94048, State: Initialized, Role: FOLLOWER
I20260807 06:37:17.256353 541 consensus_queue.cc:260] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.256353 540 consensus_queue.cc:260] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.256637 540 ts_tablet_manager.cc:1428] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 06:37:17.256637 541 ts_tablet_manager.cc:1428] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:37:17.256804 32763 heartbeater.cc:503] Master 127.2.243.62:43035 was elected leader, sending a full tablet report...
I20260807 06:37:17.256865 32537 heartbeater.cc:503] Master 127.2.243.62:43035 was elected leader, sending a full tablet report...
W20260807 06:37:17.311151 32764 tablet.cc:2369] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:37:17.313977 547 raft_consensus.cc:482] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:17.314087 547 raft_consensus.cc:504] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.314581 547 leader_election.cc:288] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31fd21e2ef674aeb85d13f8758dbe696 (127.2.243.2:36331), 87768a4d39744cbf8460b4d15e7b245d (127.2.243.3:38985)
I20260807 06:37:17.317652 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.314676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:36331 (local address 127.2.243.1:49345)
0807 06:37:17.314730 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.314741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.314783 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.314938 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.314939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.314941 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.315058 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.315060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.315809 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.315810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.316695 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.316698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.317341 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.317342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.317421 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.317425 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.317490 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.317532 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":3}
I20260807 06:37:17.317652 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.314770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:49345 (local address 127.2.243.2:36331)
0807 06:37:17.314815 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.314817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.314826 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:17.314851 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.314852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.314864 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.314912 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.315155 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.315725 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.316779 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.317240 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.317313 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.317322 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.317501 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.317584 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:37:17.317894 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.314874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:50109 (local address 127.2.243.3:38985)
0807 06:37:17.314918 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.314921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.315083 (+ 162us) server_negotiation.cc:407] Connection header received
0807 06:37:17.315201 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.315202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.315215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.315243 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.315446 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.315969 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.316996 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.317514 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.317578 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.317587 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.317781 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.317826 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:37:17.318068 32678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b06476f9a747c39423cdc9c4c1f5fe" candidate_uuid: "a30e5602c6af4b33ba899b24d2c94048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fd21e2ef674aeb85d13f8758dbe696" is_pre_election: true
I20260807 06:37:17.318133 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.314722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38985 (local address 127.2.243.1:50109)
0807 06:37:17.314984 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.314993 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.315096 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.315303 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.315304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.315305 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.315369 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.315371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.316062 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.316063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.316909 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.316912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.317600 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.317601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.317674 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.317679 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.318028 (+ 349us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.318072 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":128,"threads_started":1}
I20260807 06:37:17.318251 32678 raft_consensus.cc:2454] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30e5602c6af4b33ba899b24d2c94048 in term 0.
I20260807 06:37:17.318527 32342 leader_election.cc:302] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31fd21e2ef674aeb85d13f8758dbe696, a30e5602c6af4b33ba899b24d2c94048; no voters:
I20260807 06:37:17.318547 440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b06476f9a747c39423cdc9c4c1f5fe" candidate_uuid: "a30e5602c6af4b33ba899b24d2c94048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87768a4d39744cbf8460b4d15e7b245d" is_pre_election: true
I20260807 06:37:17.318687 440 raft_consensus.cc:2454] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30e5602c6af4b33ba899b24d2c94048 in term 0.
I20260807 06:37:17.318686 547 raft_consensus.cc:2781] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:37:17.318739 547 raft_consensus.cc:482] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:37:17.318766 547 raft_consensus.cc:3037] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:17.319500 547 raft_consensus.cc:504] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.319650 547 leader_election.cc:288] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [CANDIDATE]: Term 1 election: Requested vote from peers 31fd21e2ef674aeb85d13f8758dbe696 (127.2.243.2:36331), 87768a4d39744cbf8460b4d15e7b245d (127.2.243.3:38985)
I20260807 06:37:17.319785 440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b06476f9a747c39423cdc9c4c1f5fe" candidate_uuid: "a30e5602c6af4b33ba899b24d2c94048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87768a4d39744cbf8460b4d15e7b245d"
I20260807 06:37:17.319792 32678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89b06476f9a747c39423cdc9c4c1f5fe" candidate_uuid: "a30e5602c6af4b33ba899b24d2c94048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fd21e2ef674aeb85d13f8758dbe696"
I20260807 06:37:17.319845 440 raft_consensus.cc:3037] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:17.319887 32678 raft_consensus.cc:3037] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:17.320633 440 raft_consensus.cc:2454] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30e5602c6af4b33ba899b24d2c94048 in term 1.
I20260807 06:37:17.320634 32678 raft_consensus.cc:2454] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30e5602c6af4b33ba899b24d2c94048 in term 1.
I20260807 06:37:17.320807 32334 leader_election.cc:302] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87768a4d39744cbf8460b4d15e7b245d, a30e5602c6af4b33ba899b24d2c94048; no voters:
I20260807 06:37:17.320909 547 raft_consensus.cc:2781] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:17.321080 547 raft_consensus.cc:686] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [term 1 LEADER]: Becoming Leader. State: Replica: a30e5602c6af4b33ba899b24d2c94048, State: Running, Role: LEADER
I20260807 06:37:17.321171 547 consensus_queue.cc:237] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } }
I20260807 06:37:17.321939 32207 catalog_manager.cc:5495] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 reported cstate change: term changed from 0 to 1, leader changed from <none> to a30e5602c6af4b33ba899b24d2c94048 (127.2.243.1). New cstate: current_term: 1 leader_uuid: "a30e5602c6af4b33ba899b24d2c94048" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a30e5602c6af4b33ba899b24d2c94048" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 } health_report { overall_health: UNKNOWN } } }
I20260807 06:37:17.338346 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.335981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.2.243.1:40529)
0807 06:37:17.336031 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.336033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.336042 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:17.336095 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.336095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.336114 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.336192 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:17.336322 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.336842 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.337596 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.337784 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.337867 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.338022 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:17.338116 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:17.338226 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.338256 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.338263 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:37:17.340201 32678 raft_consensus.cc:1261] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 1 FOLLOWER]: Refusing update from remote peer a30e5602c6af4b33ba899b24d2c94048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:17.340261 440 raft_consensus.cc:1261] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 1 FOLLOWER]: Refusing update from remote peer a30e5602c6af4b33ba899b24d2c94048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:17.340628 547 consensus_queue.cc:1035] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31fd21e2ef674aeb85d13f8758dbe696" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 36331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:17.340837 550 consensus_queue.cc:1035] T 89b06476f9a747c39423cdc9c4c1f5fe P a30e5602c6af4b33ba899b24d2c94048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87768a4d39744cbf8460b4d15e7b245d" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:17.343302 554 mvcc.cc:205] Tried to move back new op lower bound from 7315802674542620672 to 7315802674467966976. Current Snapshot: MvccSnapshot[applied={T|T < 7315802674542620672}]
I20260807 06:37:17.343868 555 mvcc.cc:205] Tried to move back new op lower bound from 7315802674542620672 to 7315802674467966976. Current Snapshot: MvccSnapshot[applied={T|T < 7315802674542620672}]
I20260807 06:37:17.345867 32207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:17.345974 32207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:17.346822 32207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7db8210bcdb946669d7c8ece4a071626: Sending DeleteTablet for 3 replicas of tablet 89b06476f9a747c39423cdc9c4c1f5fe
I20260807 06:37:17.347178 32647 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b06476f9a747c39423cdc9c4c1f5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:17 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:39848
I20260807 06:37:17.347182 32424 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b06476f9a747c39423cdc9c4c1f5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:17 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:60362
I20260807 06:37:17.347416 405 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b06476f9a747c39423cdc9c4c1f5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:17 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:53778
I20260807 06:37:17.347433 565 tablet_replica.cc:330] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: stopping tablet replica
I20260807 06:37:17.347545 565 raft_consensus.cc:2229] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:17.347641 565 raft_consensus.cc:2258] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:17.347635 569 tablet_replica.cc:330] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: stopping tablet replica
I20260807 06:37:17.347721 569 raft_consensus.cc:2229] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:17.347812 569 raft_consensus.cc:2258] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:17.347833 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 32308
I20260807 06:37:17.348765 565 ts_tablet_manager.cc:1905] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:17.353718 565 ts_tablet_manager.cc:1918] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:17.353811 565 log.cc:1196] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/89b06476f9a747c39423cdc9c4c1f5fe
I20260807 06:37:17.354204 565 ts_tablet_manager.cc:1939] T 89b06476f9a747c39423cdc9c4c1f5fe P 31fd21e2ef674aeb85d13f8758dbe696: Deleting consensus metadata
I20260807 06:37:17.354745 32187 catalog_manager.cc:4841] TS 31fd21e2ef674aeb85d13f8758dbe696 (127.2.243.2:36331): tablet 89b06476f9a747c39423cdc9c4c1f5fe (table test-table [id=b946e19c5db24caa873dbd9372f52150]) successfully deleted
I20260807 06:37:17.355289 569 ts_tablet_manager.cc:1905] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:17.357683 569 ts_tablet_manager.cc:1918] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:17.357771 569 log.cc:1196] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/89b06476f9a747c39423cdc9c4c1f5fe
I20260807 06:37:17.358165 569 ts_tablet_manager.cc:1939] T 89b06476f9a747c39423cdc9c4c1f5fe P 87768a4d39744cbf8460b4d15e7b245d: Deleting consensus metadata
I20260807 06:37:17.358667 32183 catalog_manager.cc:4841] TS 87768a4d39744cbf8460b4d15e7b245d (127.2.243.3:38985): tablet 89b06476f9a747c39423cdc9c4c1f5fe (table test-table [id=b946e19c5db24caa873dbd9372f52150]) successfully deleted
W20260807 06:37:17.366648 32182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:40529 (error 108)
I20260807 06:37:17.366894 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 32544
I20260807 06:37:17.368762 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.367074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:40529 (local address 127.0.0.1:60384)
0807 06:37:17.368682 (+ 1608us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.368722 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1587}
W20260807 06:37:17.369747 32182 catalog_manager.cc:4584] TS a30e5602c6af4b33ba899b24d2c94048 (127.2.243.1:40529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89b06476f9a747c39423cdc9c4c1f5fe: Network error: Client connection negotiation failed: client connection to 127.2.243.1:40529: connect: Connection refused (error 111)
I20260807 06:37:17.380483 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 301
I20260807 06:37:17.390113 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 32157
2026-08-07T06:37:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2171 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T06:37:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:17Z Disabled control of system clock
I20260807 06:37:17.424243 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:34417
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:40981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:34417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:17.506704 580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:17.506978 580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:17.507014 580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:17.508194 580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:17.508270 580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:17.508292 580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:17.508313 580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:17.508344 580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:17.509761 580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:40981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:34417
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:34417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:17.510078 580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:17.510393 580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:17.515513 592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.515519 588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.515513 589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.515786 580 server_base.cc:1034] running on GCE node
I20260807 06:37:17.516038 580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:17.516306 580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:17.517478 580 hybrid_clock.cc:648] HybridClock initialized: now 1786084637517431 us; error 30 us; skew 500 ppm
I20260807 06:37:17.518764 580 webserver.cc:460] Webserver started at http://127.2.243.62:42073/ using document root <none> and password file <none>
I20260807 06:37:17.519033 580 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:17.519096 580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:17.519236 580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:17.520295 580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "f3a3975c3cba429fbdda8f3773330e50"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "b7e9c6a2b42cb28a9ab20eda2a735270"
server_key_iv: "90eda8686b81bf963931a3926befcdc6"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.520685 580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "f3a3975c3cba429fbdda8f3773330e50"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "b7e9c6a2b42cb28a9ab20eda2a735270"
server_key_iv: "90eda8686b81bf963931a3926befcdc6"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.522034 580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:17.522791 598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.522964 580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:37:17.523056 580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "f3a3975c3cba429fbdda8f3773330e50"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "b7e9c6a2b42cb28a9ab20eda2a735270"
server_key_iv: "90eda8686b81bf963931a3926befcdc6"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.523146 580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:17.534404 580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:17.534710 580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:17.534861 580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:17.539006 580 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:34417
I20260807 06:37:17.539052 685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:34417 every 8 connection(s)
I20260807 06:37:17.539376 580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:17.540302 686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:17.542721 686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50: Bootstrap starting.
I20260807 06:37:17.543371 686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:17.543681 686 log.cc:824] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:17.544375 686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50: No bootstrap required, opened a new log
I20260807 06:37:17.545991 686 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a3975c3cba429fbdda8f3773330e50" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34417 } }
I20260807 06:37:17.546154 686 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:17.546205 686 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a3975c3cba429fbdda8f3773330e50, State: Initialized, Role: FOLLOWER
I20260807 06:37:17.546351 686 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a3975c3cba429fbdda8f3773330e50" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34417 } }
I20260807 06:37:17.546437 686 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:17.546480 686 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:17.546531 686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:17.547039 686 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a3975c3cba429fbdda8f3773330e50" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34417 } }
I20260807 06:37:17.547163 686 leader_election.cc:302] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3a3975c3cba429fbdda8f3773330e50; no voters:
I20260807 06:37:17.547348 686 leader_election.cc:288] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:17.547402 689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:17.547614 686 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:17.547633 689 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [term 1 LEADER]: Becoming Leader. State: Replica: f3a3975c3cba429fbdda8f3773330e50, State: Running, Role: LEADER
I20260807 06:37:17.547876 689 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a3975c3cba429fbdda8f3773330e50" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34417 } }
I20260807 06:37:17.548307 690 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3a3975c3cba429fbdda8f3773330e50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a3975c3cba429fbdda8f3773330e50" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34417 } } }
I20260807 06:37:17.548400 691 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a3975c3cba429fbdda8f3773330e50. Latest consensus state: current_term: 1 leader_uuid: "f3a3975c3cba429fbdda8f3773330e50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a3975c3cba429fbdda8f3773330e50" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 34417 } } }
I20260807 06:37:17.548465 690 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:17.548760 691 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:17.548789 701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:17.549139 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 580
I20260807 06:37:17.549278 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:17.549432 701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:17.549482 3020 external_mini_cluster.cc:1428] Setting key 9dc3ec889e0698a0b09824f00059785a
I20260807 06:37:17.551034 701 catalog_manager.cc:1347] Generated new cluster ID: 75dc3ed4900444949c4088603fb952c1
I20260807 06:37:17.551087 701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:17.556285 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.549976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.2.243.62:34417)
0807 06:37:17.550481 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.550489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.550519 (+ 30us) server_negotiation.cc:407] Connection header received
0807 06:37:17.550649 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.550655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.550737 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.550881 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.551153 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.552522 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.553816 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.554465 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.554943 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.555359 (+ 416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.555363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.555363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:17.555583 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.555628 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:17.555642 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.555946 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.555978 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.556069 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":77,"threads_started":1}
I20260807 06:37:17.561369 701 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:17.561866 701 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:17.569263 701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50: Generated new TSK 0
I20260807 06:37:17.569442 701 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:17.576409 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:34417
--builtin_ntp_servers=127.2.243.20:40981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:17.663739 723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:17.663954 723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:17.663980 723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:17.665148 723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:17.665194 723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:17.665220 723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:37:17.666517 723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:40981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:17.666755 723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:17.667068 723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:17.672960 732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.673008 731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.672957 734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.673553 723 server_base.cc:1034] running on GCE node
I20260807 06:37:17.673755 723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:17.674022 723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:17.675189 723 hybrid_clock.cc:648] HybridClock initialized: now 1786084637675175 us; error 37 us; skew 500 ppm
I20260807 06:37:17.676599 723 webserver.cc:460] Webserver started at http://127.2.243.1:45599/ using document root <none> and password file <none>
I20260807 06:37:17.676874 723 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:17.676947 723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:17.677104 723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:17.678251 723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "6246f06e972948a3b2dcff893f024804"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "1a29997147f8475c38185182e2ce88ce"
server_key_iv: "41d4bc3e20e720a49802411f27012047"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.678686 723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "6246f06e972948a3b2dcff893f024804"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "1a29997147f8475c38185182e2ce88ce"
server_key_iv: "41d4bc3e20e720a49802411f27012047"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.680150 723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:17.681041 742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.681222 723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.681339 723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "6246f06e972948a3b2dcff893f024804"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "1a29997147f8475c38185182e2ce88ce"
server_key_iv: "41d4bc3e20e720a49802411f27012047"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.681420 723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:17.703890 723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:17.704232 723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:17.704401 723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:17.704686 723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:17.705205 723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:17.705272 723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.705312 723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:17.705343 723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.713495 723 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:43483
I20260807 06:37:17.713915 723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:37:17.714738 937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:43483 every 8 connection(s)
I20260807 06:37:17.722020 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.715790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:40555 (local address 127.2.243.62:34417)
0807 06:37:17.715846 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.715848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.715904 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:37:17.716763 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.716764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.716813 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.716918 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.717953 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.718701 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.720234 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.720621 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.720708 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.721714 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.721716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.721716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:17.721721 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.721737 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:17.721742 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.721802 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.721904 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.721934 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:37:17.722242 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.715429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:34417 (local address 127.2.243.1:40555)
0807 06:37:17.715695 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.715740 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.716613 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.717074 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.717079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.717090 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:17.717825 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.717832 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.718831 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.718831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.720118 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.720121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.720736 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.720737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.720883 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.721529 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:17.721530 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:17.721570 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:17.721599 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:17.721808 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:17.721813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:17.721816 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.721903 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.722013 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":118,"threads_started":1}
I20260807 06:37:17.722546 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 723
I20260807 06:37:17.722711 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:37:17.722782 941 heartbeater.cc:347] Connected to a master server at 127.2.243.62:34417
I20260807 06:37:17.722882 941 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:17.722968 3020 external_mini_cluster.cc:1428] Setting key 3003b35b6dd26d7612327ba8c8e4a2e4
I20260807 06:37:17.723090 941 heartbeater.cc:511] Master 127.2.243.62:34417 requested a full tablet report, sending...
I20260807 06:37:17.723625 623 ts_manager.cc:194] Registered new tserver with Master: 6246f06e972948a3b2dcff893f024804 (127.2.243.1:43483)
I20260807 06:37:17.724300 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:34417
--builtin_ntp_servers=127.2.243.20:40981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:37:17.724465 623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.1:40555
W20260807 06:37:17.807623 947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:17.807874 947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:17.807910 947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:17.809042 947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:17.809109 947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:17.809150 947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:37:17.810535 947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:40981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:17.810811 947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:17.811120 947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:17.816300 963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.816362 958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.816560 947 server_base.cc:1034] running on GCE node
W20260807 06:37:17.816308 967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.816900 947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:17.817138 947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:17.818356 947 hybrid_clock.cc:648] HybridClock initialized: now 1786084637818337 us; error 44 us; skew 500 ppm
I20260807 06:37:17.819806 947 webserver.cc:460] Webserver started at http://127.2.243.2:46827/ using document root <none> and password file <none>
I20260807 06:37:17.820032 947 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:17.820079 947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:17.820178 947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:17.821326 947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "8b53a6dcc0424df3a121400fbcde09b9"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "18a8c06871db6652dce9d3d22ed1b6fc"
server_key_iv: "d7f28274d2cf024ba319a14a467cf83d"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.821662 947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "8b53a6dcc0424df3a121400fbcde09b9"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "18a8c06871db6652dce9d3d22ed1b6fc"
server_key_iv: "d7f28274d2cf024ba319a14a467cf83d"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.823024 947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:37:17.823788 978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.823973 947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:37:17.824065 947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "8b53a6dcc0424df3a121400fbcde09b9"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "18a8c06871db6652dce9d3d22ed1b6fc"
server_key_iv: "d7f28274d2cf024ba319a14a467cf83d"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.824122 947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:17.840149 947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:17.840436 947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:17.840680 947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:17.840921 947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:17.841292 947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:17.841331 947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.841358 947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:17.841373 947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.848094 947 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:34483
I20260807 06:37:17.848194 1159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:34483 every 8 connection(s)
I20260807 06:37:17.848623 947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:37:17.850000 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 947
I20260807 06:37:17.850103 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:37:17.850312 3020 external_mini_cluster.cc:1428] Setting key 3282ea425bf14c78f6c3f9f804fb9cd6
I20260807 06:37:17.851513 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:34417
--builtin_ntp_servers=127.2.243.20:40981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:37:17.855333 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.849499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:54941 (local address 127.2.243.62:34417)
0807 06:37:17.849668 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.849671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.849908 (+ 237us) server_negotiation.cc:407] Connection header received
0807 06:37:17.850642 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.850644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.850659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.850757 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.851682 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.852278 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.853643 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.853988 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.854089 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.855005 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.855007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.855007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:17.855013 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.855031 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:17.855036 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.855094 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.855187 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.855216 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 06:37:17.855484 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.849518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:34417 (local address 127.2.243.2:54941)
0807 06:37:17.849727 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.849762 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.850520 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.850816 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.850819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.850828 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:17.851551 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.851558 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.852396 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.852397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.853520 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.853523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.854096 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.854096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.854207 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.854849 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:17.854850 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:17.854882 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:17.854907 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:17.855105 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:17.855110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:17.855113 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.855171 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.855258 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":121,"threads_started":1}
I20260807 06:37:17.855998 1160 heartbeater.cc:347] Connected to a master server at 127.2.243.62:34417
I20260807 06:37:17.856102 1160 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:17.856304 1160 heartbeater.cc:511] Master 127.2.243.62:34417 requested a full tablet report, sending...
I20260807 06:37:17.856745 623 ts_manager.cc:194] Registered new tserver with Master: 8b53a6dcc0424df3a121400fbcde09b9 (127.2.243.2:34483)
I20260807 06:37:17.857229 623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.2:54941
W20260807 06:37:17.936520 1164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:17.936749 1164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:17.936774 1164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:17.937999 1164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:17.938045 1164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:17.938071 1164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:37:17.939399 1164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:40981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:17.939640 1164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:17.939951 1164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:17.945276 1178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.945276 1175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:17.945292 1176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:17.945691 1164 server_base.cc:1034] running on GCE node
I20260807 06:37:17.945935 1164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:17.946171 1164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:17.947351 1164 hybrid_clock.cc:648] HybridClock initialized: now 1786084637947344 us; error 36 us; skew 500 ppm
I20260807 06:37:17.948976 1164 webserver.cc:460] Webserver started at http://127.2.243.3:34285/ using document root <none> and password file <none>
I20260807 06:37:17.949247 1164 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:17.949345 1164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:17.949522 1164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:17.950636 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "72da3a879b334fb9b07a87e5af254309"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "5403f1ff89c7ef9cd5ce777d5df2ae7b"
server_key_iv: "9669828a968c6535a3cf17438c968647"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.951048 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "72da3a879b334fb9b07a87e5af254309"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "5403f1ff89c7ef9cd5ce777d5df2ae7b"
server_key_iv: "9669828a968c6535a3cf17438c968647"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.952440 1164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:17.953392 1187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.953600 1164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:17.953718 1164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "72da3a879b334fb9b07a87e5af254309"
format_stamp: "Formatted at 2026-08-07 06:37:17 on dist-test-slave-rwrg"
server_key: "5403f1ff89c7ef9cd5ce777d5df2ae7b"
server_key_iv: "9669828a968c6535a3cf17438c968647"
server_key_version: "encryptionkey@0"
I20260807 06:37:17.953799 1164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:17.971781 1164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:17.972157 1164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:17.972334 1164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:17.972685 1164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:17.973078 1164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:17.973141 1164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.973192 1164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:17.973229 1164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:17.979626 1164 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:42347
I20260807 06:37:17.979669 1363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:42347 every 8 connection(s)
I20260807 06:37:17.980033 1164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:37:17.985853 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.980966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:55863 (local address 127.2.243.62:34417)
0807 06:37:17.981044 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.981047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.981283 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:37:17.981854 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.981855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.981896 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.981985 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.982696 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.983225 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.984291 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.984524 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.984601 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.985544 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.985546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.985546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:17.985552 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.985567 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:17.985571 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.985642 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.985740 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.985771 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:37:17.986073 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.980890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:34417 (local address 127.2.243.3:55863)
0807 06:37:17.981157 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.981191 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.981738 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.982081 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.982085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.982094 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:17.982594 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.982598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.983325 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.983326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.984199 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.984201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.984620 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.984622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:17.984751 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.985369 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:17.985370 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:17.985398 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:17.985422 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:17.985643 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:17.985650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:17.985653 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:17.985741 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 06:37:17.985842 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":82,"threads_started":1}
I20260807 06:37:17.986577 1366 heartbeater.cc:347] Connected to a master server at 127.2.243.62:34417
I20260807 06:37:17.986657 1366 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:17.986799 1366 heartbeater.cc:511] Master 127.2.243.62:34417 requested a full tablet report, sending...
I20260807 06:37:17.987134 623 ts_manager.cc:194] Registered new tserver with Master: 72da3a879b334fb9b07a87e5af254309 (127.2.243.3:42347)
I20260807 06:37:17.987512 623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.3:55863
I20260807 06:37:17.988233 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 1164
I20260807 06:37:17.988330 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:37:17.988576 3020 external_mini_cluster.cc:1428] Setting key 7e29dbd5a3edc5b6ffe45d5777d88451
I20260807 06:37:17.989401 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:37:17.993458 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.990752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.2.243.62:34417)
0807 06:37:17.990794 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.990796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.990917 (+ 121us) server_negotiation.cc:407] Connection header received
0807 06:37:17.990951 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.990952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.990961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.991010 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:17.991150 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.991612 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.992343 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:17.992565 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:17.992615 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:17.993217 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:17.993218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:17.993218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:17.993222 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:17.993234 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:17.993237 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:17.993302 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:17.993362 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:37:17.993374 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:37:17.995200 623 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:37:17.996757 623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:37:18.003268 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.998956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:43483 (local address 127.0.0.1:47010)
0807 06:37:17.999119 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.999160 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.999226 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.999649 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.999652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.999655 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.999741 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.999743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.001080 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.001081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.002112 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:18.002114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.002971 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.002972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.003060 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.003074 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 06:37:18.003147 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 06:37:18.003196 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260807 06:37:18.003311 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.999287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.2.243.1:43483)
0807 06:37:17.999464 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.999468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.999477 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:17.999521 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.999523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.999538 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.999824 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.999886 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.000979 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.002184 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.002868 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.002965 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.003040 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:18.003175 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 06:37:18.003225 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260807 06:37:18.003402 1383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.999069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.2.243.2:34483)
0807 06:37:17.999287 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.999292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.999495 (+ 203us) server_negotiation.cc:407] Connection header received
0807 06:37:17.999670 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.999673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.999688 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.999737 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.999951 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.000693 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.002113 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.002854 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.002934 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.003028 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:18.003271 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 06:37:18.003313 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260807 06:37:18.003623 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.998956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:34483 (local address 127.0.0.1:50056)
0807 06:37:17.999423 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.999435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.999500 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.999823 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.999824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.999825 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.999864 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.999866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.000773 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.000774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.002030 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:18.002033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.003075 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.003076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.003172 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.003175 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:18.003496 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 06:37:18.003539 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":26,"thread_start_us":65,"threads_started":1}
I20260807 06:37:18.004051 802 tablet_service.cc:1467] Processing CreateTablet for tablet ebae2d420d9c48ae9b34c0780bc04da4 (DEFAULT_TABLE table=test-table [id=4bfa71fa935f4892bd743e7b975cc15e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:18.004433 802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebae2d420d9c48ae9b34c0780bc04da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:18.004630 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.999156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:42347 (local address 127.0.0.1:50772)
0807 06:37:17.999458 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:17.999467 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:17.999512 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:17.999766 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:17.999766 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:17.999768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:17.999814 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:17.999817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.002197 (+ 2380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.002197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.002798 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:18.002799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.004276 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.004280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.004358 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.004361 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:18.004454 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 06:37:18.004565 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":61,"threads_started":1}
I20260807 06:37:18.004624 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:17.999255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.2.243.3:42347)
0807 06:37:17.999420 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:17.999425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:17.999570 (+ 145us) server_negotiation.cc:407] Connection header received
0807 06:37:17.999613 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:17.999616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:17.999631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:17.999701 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:17.999888 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.000971 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.002898 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.004107 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.004195 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.004316 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:18.004471 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:37:18.004543 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260807 06:37:18.005208 1253 tablet_service.cc:1467] Processing CreateTablet for tablet ebae2d420d9c48ae9b34c0780bc04da4 (DEFAULT_TABLE table=test-table [id=4bfa71fa935f4892bd743e7b975cc15e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:18.005504 1253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebae2d420d9c48ae9b34c0780bc04da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:18.007212 1389 tablet_bootstrap.cc:492] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804: Bootstrap starting.
I20260807 06:37:18.007928 1389 tablet_bootstrap.cc:654] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:18.007983 1049 tablet_service.cc:1467] Processing CreateTablet for tablet ebae2d420d9c48ae9b34c0780bc04da4 (DEFAULT_TABLE table=test-table [id=4bfa71fa935f4892bd743e7b975cc15e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:18.008292 1389 log.cc:824] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:18.008337 1049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebae2d420d9c48ae9b34c0780bc04da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:18.008760 1391 tablet_bootstrap.cc:492] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Bootstrap starting.
I20260807 06:37:18.009148 1389 tablet_bootstrap.cc:492] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804: No bootstrap required, opened a new log
I20260807 06:37:18.009222 1389 ts_tablet_manager.cc:1397] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:37:18.009446 1391 tablet_bootstrap.cc:654] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:18.009795 1391 log.cc:824] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:18.010874 1391 tablet_bootstrap.cc:492] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: No bootstrap required, opened a new log
I20260807 06:37:18.010943 1391 ts_tablet_manager.cc:1397] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:37:18.011214 1394 tablet_bootstrap.cc:492] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Bootstrap starting.
I20260807 06:37:18.011814 1394 tablet_bootstrap.cc:654] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:18.012137 1394 log.cc:824] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:18.012008 1389 raft_consensus.cc:348] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.012205 1389 raft_consensus.cc:374] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:18.012249 1389 raft_consensus.cc:729] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6246f06e972948a3b2dcff893f024804, State: Initialized, Role: FOLLOWER
I20260807 06:37:18.012403 1389 consensus_queue.cc:260] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.012698 1389 ts_tablet_manager.cc:1428] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 06:37:18.012748 941 heartbeater.cc:503] Master 127.2.243.62:34417 was elected leader, sending a full tablet report...
I20260807 06:37:18.013038 1394 tablet_bootstrap.cc:492] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: No bootstrap required, opened a new log
I20260807 06:37:18.013113 1394 ts_tablet_manager.cc:1397] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:37:18.014569 1391 raft_consensus.cc:348] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.014711 1391 raft_consensus.cc:374] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:18.014757 1391 raft_consensus.cc:729] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72da3a879b334fb9b07a87e5af254309, State: Initialized, Role: FOLLOWER
I20260807 06:37:18.014865 1391 consensus_queue.cc:260] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.015144 1391 ts_tablet_manager.cc:1428] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 06:37:18.015195 1366 heartbeater.cc:503] Master 127.2.243.62:34417 was elected leader, sending a full tablet report...
I20260807 06:37:18.015242 1394 raft_consensus.cc:348] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.015386 1394 raft_consensus.cc:374] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:18.015426 1394 raft_consensus.cc:729] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b53a6dcc0424df3a121400fbcde09b9, State: Initialized, Role: FOLLOWER
I20260807 06:37:18.015563 1394 consensus_queue.cc:260] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.015847 1394 ts_tablet_manager.cc:1428] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:37:18.015899 1160 heartbeater.cc:503] Master 127.2.243.62:34417 was elected leader, sending a full tablet report...
I20260807 06:37:18.031104 1401 raft_consensus.cc:482] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:18.031204 1401 raft_consensus.cc:504] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.031489 1401 leader_election.cc:288] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6246f06e972948a3b2dcff893f024804 (127.2.243.1:43483), 72da3a879b334fb9b07a87e5af254309 (127.2.243.3:42347)
I20260807 06:37:18.034554 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:18.031717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:43483 (local address 127.2.243.2:53155)
0807 06:37:18.031792 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:18.031806 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:18.031871 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:18.032088 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:18.032090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:18.032091 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:18.032166 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:18.032168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.032901 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.032902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.033703 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:18.033705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.034311 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.034312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.034381 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.034385 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:37:18.034440 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 06:37:18.034482 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 06:37:18.034626 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:18.031859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:53155 (local address 127.2.243.1:43483)
0807 06:37:18.031933 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:18.031936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:18.031945 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:18.031975 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:18.031976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:18.031989 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:18.032060 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:18.032249 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.032822 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.033805 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.034226 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.034282 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.034290 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:18.034462 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 06:37:18.034500 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:37:18.034729 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:18.031825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:42347 (local address 127.2.243.2:34217)
0807 06:37:18.031988 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:18.031998 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:18.032063 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:18.032194 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:18.032195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:18.032196 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:18.032287 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:18.032288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.033035 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.033036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.033865 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:18.033868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.034457 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.034458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:18.034537 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.034541 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:37:18.034603 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:37:18.034640 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260807 06:37:18.034761 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:18.031899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:34217 (local address 127.2.243.3:42347)
0807 06:37:18.031943 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:18.031944 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:18.032076 (+ 132us) server_negotiation.cc:407] Connection header received
0807 06:37:18.032110 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:18.032110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:18.032118 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:18.032182 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:18.032370 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.032943 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.033956 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.034351 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.034425 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.034435 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:18.034632 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 06:37:18.034679 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 06:37:18.034956 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebae2d420d9c48ae9b34c0780bc04da4" candidate_uuid: "8b53a6dcc0424df3a121400fbcde09b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6246f06e972948a3b2dcff893f024804" is_pre_election: true
I20260807 06:37:18.035063 1286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebae2d420d9c48ae9b34c0780bc04da4" candidate_uuid: "8b53a6dcc0424df3a121400fbcde09b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72da3a879b334fb9b07a87e5af254309" is_pre_election: true
I20260807 06:37:18.035116 861 raft_consensus.cc:2454] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b53a6dcc0424df3a121400fbcde09b9 in term 0.
I20260807 06:37:18.035199 1286 raft_consensus.cc:2454] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b53a6dcc0424df3a121400fbcde09b9 in term 0.
I20260807 06:37:18.035380 979 leader_election.cc:302] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6246f06e972948a3b2dcff893f024804, 8b53a6dcc0424df3a121400fbcde09b9; no voters:
I20260807 06:37:18.035534 1401 raft_consensus.cc:2781] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:37:18.035624 1401 raft_consensus.cc:482] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:37:18.035663 1401 raft_consensus.cc:3037] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:18.036443 1401 raft_consensus.cc:504] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.036638 1401 leader_election.cc:288] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [CANDIDATE]: Term 1 election: Requested vote from peers 6246f06e972948a3b2dcff893f024804 (127.2.243.1:43483), 72da3a879b334fb9b07a87e5af254309 (127.2.243.3:42347)
I20260807 06:37:18.036862 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebae2d420d9c48ae9b34c0780bc04da4" candidate_uuid: "8b53a6dcc0424df3a121400fbcde09b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6246f06e972948a3b2dcff893f024804"
I20260807 06:37:18.036885 1286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebae2d420d9c48ae9b34c0780bc04da4" candidate_uuid: "8b53a6dcc0424df3a121400fbcde09b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72da3a879b334fb9b07a87e5af254309"
I20260807 06:37:18.036947 1286 raft_consensus.cc:3037] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:18.036970 861 raft_consensus.cc:3037] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:18.037710 1286 raft_consensus.cc:2454] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b53a6dcc0424df3a121400fbcde09b9 in term 1.
I20260807 06:37:18.037902 861 raft_consensus.cc:2454] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b53a6dcc0424df3a121400fbcde09b9 in term 1.
I20260807 06:37:18.037993 979 leader_election.cc:302] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72da3a879b334fb9b07a87e5af254309, 8b53a6dcc0424df3a121400fbcde09b9; no voters:
I20260807 06:37:18.038100 1401 raft_consensus.cc:2781] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:18.038280 1401 raft_consensus.cc:686] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 1 LEADER]: Becoming Leader. State: Replica: 8b53a6dcc0424df3a121400fbcde09b9, State: Running, Role: LEADER
I20260807 06:37:18.038381 1401 consensus_queue.cc:237] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } }
I20260807 06:37:18.039407 620 catalog_manager.cc:5495] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b53a6dcc0424df3a121400fbcde09b9 (127.2.243.2). New cstate: current_term: 1 leader_uuid: "8b53a6dcc0424df3a121400fbcde09b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b53a6dcc0424df3a121400fbcde09b9" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 34483 } health_report { overall_health: HEALTHY } } }
I20260807 06:37:18.052139 1383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:18.050003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.2.243.2:34483)
0807 06:37:18.050075 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:18.050077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:18.050103 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:37:18.050131 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:18.050132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:18.050149 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:18.050200 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:18.050313 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.050763 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.051464 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:18.051639 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:18.051688 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:18.051807 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:18.051888 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:18.051953 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:18.052000 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 06:37:18.052007 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:37:18.053910 861 raft_consensus.cc:1261] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 1 FOLLOWER]: Refusing update from remote peer 8b53a6dcc0424df3a121400fbcde09b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:18.053944 1286 raft_consensus.cc:1261] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 1 FOLLOWER]: Refusing update from remote peer 8b53a6dcc0424df3a121400fbcde09b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:18.054315 1403 consensus_queue.cc:1035] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72da3a879b334fb9b07a87e5af254309" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 42347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:18.054419 1401 consensus_queue.cc:1035] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6246f06e972948a3b2dcff893f024804" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 43483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:18.056941 1411 mvcc.cc:205] Tried to move back new op lower bound from 7315802677466402816 to 7315802677406187520. Current Snapshot: MvccSnapshot[applied={T|T < 7315802677466402816}]
I20260807 06:37:18.058917 620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:18.059017 620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:18.059819 620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3a3975c3cba429fbdda8f3773330e50: Sending DeleteTablet for 3 replicas of tablet ebae2d420d9c48ae9b34c0780bc04da4
I20260807 06:37:18.060163 1049 tablet_service.cc:1514] Processing DeleteTablet for tablet ebae2d420d9c48ae9b34c0780bc04da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:18 UTC) from {username='slave'} at 127.0.0.1:50056
I20260807 06:37:18.060322 802 tablet_service.cc:1514] Processing DeleteTablet for tablet ebae2d420d9c48ae9b34c0780bc04da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:18 UTC) from {username='slave'} at 127.0.0.1:47010
I20260807 06:37:18.060325 1253 tablet_service.cc:1514] Processing DeleteTablet for tablet ebae2d420d9c48ae9b34c0780bc04da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:18 UTC) from {username='slave'} at 127.0.0.1:50772
I20260807 06:37:18.060585 1427 tablet_replica.cc:330] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: stopping tablet replica
I20260807 06:37:18.060635 1426 tablet_replica.cc:330] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804: stopping tablet replica
I20260807 06:37:18.060686 1427 raft_consensus.cc:2229] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:18.060732 1426 raft_consensus.cc:2229] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:18.060770 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 723
I20260807 06:37:18.060791 1427 raft_consensus.cc:2258] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:18.060828 1426 raft_consensus.cc:2258] T ebae2d420d9c48ae9b34c0780bc04da4 P 6246f06e972948a3b2dcff893f024804 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:18.060966 1425 tablet_replica.cc:330] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: stopping tablet replica
I20260807 06:37:18.061045 1425 raft_consensus.cc:2229] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:37:18.061128 1425 raft_consensus.cc:2258] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:18.063740 1427 ts_tablet_manager.cc:1905] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:18.063787 1425 ts_tablet_manager.cc:1905] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:18.065358 1427 ts_tablet_manager.cc:1918] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:18.065423 1427 log.cc:1196] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/ebae2d420d9c48ae9b34c0780bc04da4
I20260807 06:37:18.065713 1427 ts_tablet_manager.cc:1939] T ebae2d420d9c48ae9b34c0780bc04da4 P 72da3a879b334fb9b07a87e5af254309: Deleting consensus metadata
I20260807 06:37:18.065970 1425 ts_tablet_manager.cc:1918] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:18.066045 1425 log.cc:1196] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/ebae2d420d9c48ae9b34c0780bc04da4
I20260807 06:37:18.066111 599 catalog_manager.cc:4841] TS 72da3a879b334fb9b07a87e5af254309 (127.2.243.3:42347): tablet ebae2d420d9c48ae9b34c0780bc04da4 (table test-table [id=4bfa71fa935f4892bd743e7b975cc15e]) successfully deleted
I20260807 06:37:18.066336 1425 ts_tablet_manager.cc:1939] T ebae2d420d9c48ae9b34c0780bc04da4 P 8b53a6dcc0424df3a121400fbcde09b9: Deleting consensus metadata
I20260807 06:37:18.066773 602 catalog_manager.cc:4841] TS 8b53a6dcc0424df3a121400fbcde09b9 (127.2.243.2:34483): tablet ebae2d420d9c48ae9b34c0780bc04da4 (table test-table [id=4bfa71fa935f4892bd743e7b975cc15e]) successfully deleted
W20260807 06:37:18.082341 599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:43483 (error 108)
I20260807 06:37:18.082589 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 947
I20260807 06:37:18.083267 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:18.082757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:43483 (local address 127.0.0.1:47016)
0807 06:37:18.083183 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:18.083233 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402}
W20260807 06:37:18.084905 599 catalog_manager.cc:4584] TS 6246f06e972948a3b2dcff893f024804 (127.2.243.1:43483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebae2d420d9c48ae9b34c0780bc04da4: Network error: Client connection negotiation failed: client connection to 127.2.243.1:43483: connect: Connection refused (error 111)
I20260807 06:37:18.088840 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 1164
I20260807 06:37:18.099509 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 580
2026-08-07T06:37:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (713 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:18 dist-test-slave-rwrg krb5kdc[1437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:18 dist-test-slave-rwrg krb5kdc[1437](info): set up 2 sockets
Aug 07 06:37:18 dist-test-slave-rwrg krb5kdc[1437](info): commencing operation
krb5kdc: starting...
W20260807 06:37:19.347936 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:19 dist-test-slave-rwrg krb5kdc[1437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:19.471448 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:36517
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:36517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:19.553893 1461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:19.554137 1461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:19.554174 1461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:19.555344 1461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:19.555416 1461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:19.555440 1461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:19.555470 1461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:19.555501 1461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:19.557026 1461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:36517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:36517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:19.557302 1461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:19.557557 1461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:19.562534 1467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:19.562577 1469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:19.562587 1475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:19.562815 1461 server_base.cc:1034] running on GCE node
I20260807 06:37:19.563174 1461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:19.563519 1461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:19.564711 1461 hybrid_clock.cc:648] HybridClock initialized: now 1786084639564689 us; error 40 us; skew 500 ppm
Aug 07 06:37:19 dist-test-slave-rwrg krb5kdc[1437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084639, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:19.567281 1461 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:19.567755 1461 webserver.cc:460] Webserver started at http://127.2.243.62:46235/ using document root <none> and password file <none>
I20260807 06:37:19.567960 1461 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:19.568002 1461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:19.568094 1461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:19.569139 1461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "3c94511e94de4c4f9b3566b59519f1d0"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "a69f09ec85e088ec2b48fc044a366b81"
server_key_iv: "8373d6d212561ca60e9340202e876164"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.569459 1461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "3c94511e94de4c4f9b3566b59519f1d0"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "a69f09ec85e088ec2b48fc044a366b81"
server_key_iv: "8373d6d212561ca60e9340202e876164"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.570779 1461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:19.571538 1483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:19.571775 1461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:19.571867 1461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "3c94511e94de4c4f9b3566b59519f1d0"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "a69f09ec85e088ec2b48fc044a366b81"
server_key_iv: "8373d6d212561ca60e9340202e876164"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.571923 1461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:19.586369 1461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:19.588251 1461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:19.588362 1461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:19.592171 1461 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:36517
I20260807 06:37:19.592228 1577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:36517 every 8 connection(s)
I20260807 06:37:19.592748 1461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:19.593739 1578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:19.596093 1578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0: Bootstrap starting.
I20260807 06:37:19.596699 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 1461
I20260807 06:37:19.596828 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:19.596856 1578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:19.597062 3020 external_mini_cluster.cc:1428] Setting key 8cb523c6afcaa2c60162d62e601c41ab
I20260807 06:37:19.597200 1578 log.cc:824] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:19.598031 1578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0: No bootstrap required, opened a new log
I20260807 06:37:19.600241 1578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c94511e94de4c4f9b3566b59519f1d0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36517 } }
I20260807 06:37:19.600418 1578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:19.600471 1578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c94511e94de4c4f9b3566b59519f1d0, State: Initialized, Role: FOLLOWER
I20260807 06:37:19.600641 1578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c94511e94de4c4f9b3566b59519f1d0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36517 } }
I20260807 06:37:19.600746 1578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:19.600793 1578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:19.600843 1578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:19.601403 1578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c94511e94de4c4f9b3566b59519f1d0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36517 } }
I20260807 06:37:19.601536 1578 leader_election.cc:302] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c94511e94de4c4f9b3566b59519f1d0; no voters:
I20260807 06:37:19.601728 1578 leader_election.cc:288] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:19.601797 1588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:19.601987 1588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3c94511e94de4c4f9b3566b59519f1d0, State: Running, Role: LEADER
I20260807 06:37:19.602018 1578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:19.602134 1588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c94511e94de4c4f9b3566b59519f1d0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36517 } }
I20260807 06:37:19.602555 1588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c94511e94de4c4f9b3566b59519f1d0. Latest consensus state: current_term: 1 leader_uuid: "3c94511e94de4c4f9b3566b59519f1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c94511e94de4c4f9b3566b59519f1d0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36517 } } }
I20260807 06:37:19.602667 1588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:19.602864 1589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c94511e94de4c4f9b3566b59519f1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c94511e94de4c4f9b3566b59519f1d0" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 36517 } } }
I20260807 06:37:19.602952 1589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:19.603132 1605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:19.603735 1605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 06:37:19 dist-test-slave-rwrg krb5kdc[1437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:19.606360 1605 catalog_manager.cc:1347] Generated new cluster ID: d6753722fc714884acc165afe0b79914
I20260807 06:37:19.606428 1605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:19.612310 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:19.597663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.2.243.62:36517)
0807 06:37:19.597852 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:19.597861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:19.597897 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:37:19.598374 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:19.598380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:19.598429 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:19.598615 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:19.599335 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.600220 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.601386 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.602170 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.604514 (+ 2344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:19.605730 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:19.605735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:19.605736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:19.605750 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:19.610415 (+ 4665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:19.610896 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:19.610900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:19.610900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:19.611031 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:19.611277 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:19.611278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:19.611278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:19.611615 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:19.611779 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:19.611909 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 06:37:19.612071 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":113,"threads_started":1}
I20260807 06:37:19.619676 1605 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:19.620528 1605 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:19.627349 1605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0: Generated new TSK 0
I20260807 06:37:19.627614 1605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.243.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
Entry for principal kudu/127.2.243.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab.
I20260807 06:37:19.663419 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36517
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:19.748148 1631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:19.748404 1631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:19.748446 1631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:19.749652 1631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:19.749697 1631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:19.749723 1631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:37:19.751087 1631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.1.keytab
--principal=kudu/127.2.243.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:19.751283 1631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:19.751466 1631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:19.756320 1642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:19.756389 1640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:19.756335 1639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:19.756609 1631 server_base.cc:1034] running on GCE node
I20260807 06:37:19.756827 1631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:19.757079 1631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:19.758260 1631 hybrid_clock.cc:648] HybridClock initialized: now 1786084639758239 us; error 41 us; skew 500 ppm
Aug 07 06:37:19 dist-test-slave-rwrg krb5kdc[1437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084639, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:19.761008 1631 init.cc:377] Logged in from keytab as kudu/127.2.243.1@KRBTEST.COM (short username kudu)
I20260807 06:37:19.761451 1631 webserver.cc:460] Webserver started at http://127.2.243.1:35811/ using document root <none> and password file <none>
I20260807 06:37:19.761703 1631 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:19.761775 1631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:19.761956 1631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:19.762944 1631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "871847ba83fc4d10ae5e1db23a4c7a62"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "94dc41d08dbb79121f929a9805ec8dca"
server_key_iv: "6a799041b4d6ea37d6ad9d14809963ac"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.763252 1631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "871847ba83fc4d10ae5e1db23a4c7a62"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "94dc41d08dbb79121f929a9805ec8dca"
server_key_iv: "6a799041b4d6ea37d6ad9d14809963ac"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.764591 1631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:37:19.765353 1652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:19.765578 1631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:19.765659 1631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "871847ba83fc4d10ae5e1db23a4c7a62"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "94dc41d08dbb79121f929a9805ec8dca"
server_key_iv: "6a799041b4d6ea37d6ad9d14809963ac"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.765709 1631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:19.781786 1631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:19.783875 1631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:19.783986 1631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:19.784183 1631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:19.784586 1631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:19.784627 1631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:19.784650 1631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:19.784698 1631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:19.790796 1631 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:36733
I20260807 06:37:19.790871 1850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:36733 every 8 connection(s)
I20260807 06:37:19.791208 1631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
Aug 07 06:37:19 dist-test-slave-rwrg krb5kdc[1437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084639, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.1@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:19.799263 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 1631
I20260807 06:37:19.799422 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:37:19.799701 3020 external_mini_cluster.cc:1428] Setting key bef66bfaa791533835b8b0b22fc6a7e0
I20260807 06:37:19.801496 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:19.791994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:45759 (local address 127.2.243.62:36517)
0807 06:37:19.792039 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:19.792041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:19.792259 (+ 218us) server_negotiation.cc:407] Connection header received
0807 06:37:19.792911 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:19.792912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:19.792924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:19.792989 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:19.793953 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.794458 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.795579 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.795795 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.795857 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:19.798266 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:19.798268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:19.798268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:19.798275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:19.799917 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:19.800384 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:19.800386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:19.800386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:19.800475 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:19.800759 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:19.800760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:19.800761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:19.800917 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:19.801002 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:19.801224 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 06:37:19.801328 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:19.801594 1853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:19.791947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:36517 (local address 127.2.243.1:45759)
0807 06:37:19.792139 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:19.792181 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:19.792753 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:19.793066 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:19.793069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:19.793672 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:19.793862 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:19.793866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.794618 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.794619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:19.795471 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:19.795473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.795888 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.795890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:19.796009 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:19.796659 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:19.796668 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:19.798155 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:19.799999 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:19.800007 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:19.800009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:19.800262 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:19.800586 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:19.800587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:19.800588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:19.800689 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:19.800986 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:19.800990 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:19.801099 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 06:37:19.801176 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:37:19.801291 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":116,"threads_started":1}
I20260807 06:37:19.802155 1851 heartbeater.cc:347] Connected to a master server at 127.2.243.62:36517
I20260807 06:37:19.802271 1851 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:19.802506 1851 heartbeater.cc:511] Master 127.2.243.62:36517 requested a full tablet report, sending...
I20260807 06:37:19.803093 1511 ts_manager.cc:194] Registered new tserver with Master: 871847ba83fc4d10ae5e1db23a4c7a62 (127.2.243.1:36733)
I20260807 06:37:19.803961 1511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.1@KRBTEST.COM'} at 127.2.243.1:45759
WARNING: no policy specified for kudu/127.2.243.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
Entry for principal kudu/127.2.243.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab.
I20260807 06:37:19.827499 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36517
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:19.910163 1856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:19.910358 1856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:19.910380 1856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:19.911413 1856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:19.911446 1856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:19.911470 1856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:37:19.912909 1856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.2.keytab
--principal=kudu/127.2.243.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:19.913089 1856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:19.913254 1856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:37:19.918176 1856 server_base.cc:1034] running on GCE node
W20260807 06:37:19.918160 1867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:19.918188 1873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:19.918149 1866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:19.918725 1856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:19.918967 1856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:19.920133 1856 hybrid_clock.cc:648] HybridClock initialized: now 1786084639920123 us; error 28 us; skew 500 ppm
Aug 07 06:37:19 dist-test-slave-rwrg krb5kdc[1437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084639, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:19.922915 1856 init.cc:377] Logged in from keytab as kudu/127.2.243.2@KRBTEST.COM (short username kudu)
I20260807 06:37:19.923389 1856 webserver.cc:460] Webserver started at http://127.2.243.2:33361/ using document root <none> and password file <none>
I20260807 06:37:19.923633 1856 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:19.923727 1856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:19.923877 1856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:19.925012 1856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "5d1819e5cdb0472eb1a72e6edc98162c"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "1a4d784d45c8a438e3b4923be55fc27f"
server_key_iv: "c4c925720bac6e82fa63e3ea70bc77ee"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.925364 1856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "5d1819e5cdb0472eb1a72e6edc98162c"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "1a4d784d45c8a438e3b4923be55fc27f"
server_key_iv: "c4c925720bac6e82fa63e3ea70bc77ee"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.926708 1856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:19.927428 1881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:19.927615 1856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:19.927723 1856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "5d1819e5cdb0472eb1a72e6edc98162c"
format_stamp: "Formatted at 2026-08-07 06:37:19 on dist-test-slave-rwrg"
server_key: "1a4d784d45c8a438e3b4923be55fc27f"
server_key_iv: "c4c925720bac6e82fa63e3ea70bc77ee"
server_key_version: "encryptionkey@0"
I20260807 06:37:19.927796 1856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:19.944187 1856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:19.946537 1856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:19.946705 1856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:19.946949 1856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:19.947264 1856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:19.947319 1856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:19.947359 1856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:19.947394 1856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:19.953361 1856 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:33989
I20260807 06:37:19.953406 2080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:33989 every 8 connection(s)
I20260807 06:37:19.953733 1856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
Aug 07 06:37:19 dist-test-slave-rwrg krb5kdc[1437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084639, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.2@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:19.963099 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 1856
I20260807 06:37:19.963286 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:37:19.963349 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:19.954464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:41175 (local address 127.2.243.62:36517)
0807 06:37:19.954533 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:19.954535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:19.954742 (+ 207us) server_negotiation.cc:407] Connection header received
0807 06:37:19.955258 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:19.955259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:19.955271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:19.955329 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:19.956276 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.956845 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.957861 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.958071 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.958155 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:19.960276 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:19.960277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:19.960277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:19.960284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:19.961613 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:19.962060 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:19.962060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:19.962061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:19.962120 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:19.962559 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:19.962560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:19.962560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:19.962700 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:19.962778 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:19.963134 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 06:37:19.963248 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:37:19.963596 3020 external_mini_cluster.cc:1428] Setting key 306752676fe28e12c99eb811cf75e855
I20260807 06:37:19.963603 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:19.954440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:36517 (local address 127.2.243.2:41175)
0807 06:37:19.954627 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:19.954661 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:19.955156 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:19.955400 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:19.955403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:19.955968 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:19.956168 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:19.956172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.956933 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.956934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:19.957765 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:19.957768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:19.958162 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:19.958163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:19.958283 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:19.958797 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:19.958805 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:19.960182 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:19.961746 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:19.961756 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:19.961758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:19.961974 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:19.962273 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:19.962275 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:19.962275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:19.962451 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:19.962782 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:19.962788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:19.962938 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 06:37:19.963096 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 06:37:19.963296 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260807 06:37:19.964246 2083 heartbeater.cc:347] Connected to a master server at 127.2.243.62:36517
I20260807 06:37:19.964378 2083 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:19.964663 2083 heartbeater.cc:511] Master 127.2.243.62:36517 requested a full tablet report, sending...
I20260807 06:37:19.965157 1510 ts_manager.cc:194] Registered new tserver with Master: 5d1819e5cdb0472eb1a72e6edc98162c (127.2.243.2:33989)
I20260807 06:37:19.965586 1510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.2@KRBTEST.COM'} at 127.2.243.2:41175
WARNING: no policy specified for kudu/127.2.243.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
Entry for principal kudu/127.2.243.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab.
I20260807 06:37:19.991622 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36517
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:20.078998 2089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:20.079200 2089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:20.079226 2089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:20.080365 2089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:20.080408 2089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:20.080433 2089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:37:20.081935 2089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.3.keytab
--principal=kudu/127.2.243.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:36517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:20.082178 2089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:20.082408 2089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:20.087337 2102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.087329 2098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.087472 2106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:20.087706 2089 server_base.cc:1034] running on GCE node
I20260807 06:37:20.087893 2089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:20.088115 2089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:20.089270 2089 hybrid_clock.cc:648] HybridClock initialized: now 1786084640089247 us; error 41 us; skew 500 ppm
Aug 07 06:37:20 dist-test-slave-rwrg krb5kdc[1437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084640, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:20.091900 2089 init.cc:377] Logged in from keytab as kudu/127.2.243.3@KRBTEST.COM (short username kudu)
I20260807 06:37:20.092268 2089 webserver.cc:460] Webserver started at http://127.2.243.3:39345/ using document root <none> and password file <none>
I20260807 06:37:20.092451 2089 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:20.092543 2089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:20.092669 2089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:20.093588 2089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "4159f1c1eb3144d682a33e4b7296789a"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "48c5f6fafe53265704107af882d370b8"
server_key_iv: "a6338fbe31c3c923578eb828d78b856e"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.093895 2089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "4159f1c1eb3144d682a33e4b7296789a"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "48c5f6fafe53265704107af882d370b8"
server_key_iv: "a6338fbe31c3c923578eb828d78b856e"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.095117 2089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:37:20.095785 2118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.095973 2089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:37:20.096050 2089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "4159f1c1eb3144d682a33e4b7296789a"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "48c5f6fafe53265704107af882d370b8"
server_key_iv: "a6338fbe31c3c923578eb828d78b856e"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.096105 2089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:20.112004 2089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:20.114354 2089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:20.114521 2089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:20.114768 2089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:20.115089 2089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:20.115123 2089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.115167 2089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:20.115195 2089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.121002 2089 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:38505
I20260807 06:37:20.121114 2307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:38505 every 8 connection(s)
I20260807 06:37:20.121397 2089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:37:20.127238 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 2089
I20260807 06:37:20.127349 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:37:20.127568 3020 external_mini_cluster.cc:1428] Setting key 62efdcd0d4790c7d2e3a50d2a8f95a92
Aug 07 06:37:20 dist-test-slave-rwrg krb5kdc[1437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084640, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.3@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:20.131095 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.122167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:48479 (local address 127.2.243.62:36517)
0807 06:37:20.122233 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.122235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.122432 (+ 197us) server_negotiation.cc:407] Connection header received
0807 06:37:20.123031 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.123032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.123046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.123112 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:20.124068 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.124655 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.125654 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.125866 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.125945 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.128294 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:20.128295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:20.128295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:20.128303 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:20.129721 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:20.130166 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:20.130167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:20.130167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:20.130216 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:20.130510 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:20.130511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:20.130512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:20.130647 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:20.130701 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.130953 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.131011 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:37:20.131282 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.122142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:36517 (local address 127.2.243.3:48479)
0807 06:37:20.122316 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.122350 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.122881 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.123188 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.123191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.123782 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:20.123967 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.123971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.124750 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.124751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.125535 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.125537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.125952 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.125953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.126078 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.126497 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:20.126505 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:20.128193 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:20.129863 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:20.129869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:20.129870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:20.130095 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:20.130303 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:20.130303 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:20.130304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:20.130433 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:20.130705 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:20.130710 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:20.130817 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.130898 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.131058 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260807 06:37:20.131743 2309 heartbeater.cc:347] Connected to a master server at 127.2.243.62:36517
I20260807 06:37:20.131822 2309 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:20.132014 2309 heartbeater.cc:511] Master 127.2.243.62:36517 requested a full tablet report, sending...
I20260807 06:37:20.132392 1510 ts_manager.cc:194] Registered new tserver with Master: 4159f1c1eb3144d682a33e4b7296789a (127.2.243.3:38505)
I20260807 06:37:20.132817 1510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.243.3@KRBTEST.COM'} at 127.2.243.3:48479
I20260807 06:37:20.139410 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:37:20.147239 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.140890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.2.243.62:36517)
0807 06:37:20.140951 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.140954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.141138 (+ 184us) server_negotiation.cc:407] Connection header received
0807 06:37:20.141186 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.141187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.141196 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.141243 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:20.141784 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.142275 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.143004 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.143184 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.143241 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.144760 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:20.144761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:20.144761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:20.144767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:20.146076 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:20.146418 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:20.146418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:20.146419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:20.146469 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:20.146715 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:20.146715 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:20.146716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:20.146853 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:20.146937 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.147084 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.147144 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 06:37:20.149132 1510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:37:20.150741 1510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:37:20.157757 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.153656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:36733 (local address 127.0.0.1:45906)
0807 06:37:20.153943 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.153953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.154005 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.154235 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.154236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.154241 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.154303 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.154304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.155283 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.155284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.156518 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.156521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.157480 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.157481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.157560 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.157566 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.157614 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.157660 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260807 06:37:20.157841 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.153489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.2.243.2:33989)
0807 06:37:20.153702 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.153707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.153996 (+ 289us) server_negotiation.cc:407] Connection header received
0807 06:37:20.154042 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.154045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.154063 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.154138 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.154565 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.155612 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.156773 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.157275 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.157389 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.157454 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.157712 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.157754 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260807 06:37:20.157855 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.153561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.2.243.3:38505)
0807 06:37:20.154178 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.154182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.154193 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:37:20.154233 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.154234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.154250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.154327 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.154430 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.155555 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.156819 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.157504 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.157574 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.157629 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.157728 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.157766 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":95,"threads_started":1}
I20260807 06:37:20.158097 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.153371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38505 (local address 127.0.0.1:43714)
0807 06:37:20.153803 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.153837 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.153904 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.154324 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.154325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.154327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.154374 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.154377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.155671 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.155672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.156740 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.156743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.157594 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.157595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.157648 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.157651 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.157713 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.157748 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":79,"threads_started":1}
I20260807 06:37:20.158192 2344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.153767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.2.243.1:36733)
0807 06:37:20.153961 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.153966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.153998 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:37:20.154073 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.154075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.154092 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.154163 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.154390 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.155167 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.156600 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.157304 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.157376 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.157440 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.158073 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.158107 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":92,"threads_started":1}
I20260807 06:37:20.158334 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.153371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:33989 (local address 127.0.0.1:50708)
0807 06:37:20.153802 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.153833 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.153913 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.154232 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.154235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.154238 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.154480 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.154482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.155720 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.155721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.156679 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.156682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.157394 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.157395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.157494 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.157509 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.157585 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.157641 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":4,"thread_start_us":93,"threads_started":1}
I20260807 06:37:20.158900 2195 tablet_service.cc:1467] Processing CreateTablet for tablet 479feee71bde49b1b0151bc39c3db8f8 (DEFAULT_TABLE table=test-table [id=1c21d3316dea4948b9276e8bfc1b23aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:20.158908 1733 tablet_service.cc:1467] Processing CreateTablet for tablet 479feee71bde49b1b0151bc39c3db8f8 (DEFAULT_TABLE table=test-table [id=1c21d3316dea4948b9276e8bfc1b23aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:20.159128 1966 tablet_service.cc:1467] Processing CreateTablet for tablet 479feee71bde49b1b0151bc39c3db8f8 (DEFAULT_TABLE table=test-table [id=1c21d3316dea4948b9276e8bfc1b23aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:20.159307 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479feee71bde49b1b0151bc39c3db8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:20.159308 2195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479feee71bde49b1b0151bc39c3db8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:20.159425 1966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479feee71bde49b1b0151bc39c3db8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:20.162173 2345 tablet_bootstrap.cc:492] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Bootstrap starting.
I20260807 06:37:20.162859 2346 tablet_bootstrap.cc:492] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62: Bootstrap starting.
I20260807 06:37:20.163053 2345 tablet_bootstrap.cc:654] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:20.163587 2346 tablet_bootstrap.cc:654] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:20.163705 2345 log.cc:824] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:20.164136 2346 log.cc:824] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:20.162634 2350 tablet_bootstrap.cc:492] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Bootstrap starting.
I20260807 06:37:20.164565 2345 tablet_bootstrap.cc:492] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: No bootstrap required, opened a new log
I20260807 06:37:20.164638 2345 ts_tablet_manager.cc:1397] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:37:20.164948 2346 tablet_bootstrap.cc:492] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62: No bootstrap required, opened a new log
I20260807 06:37:20.165019 2346 ts_tablet_manager.cc:1397] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:37:20.165067 2350 tablet_bootstrap.cc:654] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:20.165663 2350 log.cc:824] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:20.166471 2350 tablet_bootstrap.cc:492] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: No bootstrap required, opened a new log
I20260807 06:37:20.166553 2350 ts_tablet_manager.cc:1397] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:37:20.166778 2346 raft_consensus.cc:348] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.166932 2346 raft_consensus.cc:374] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:20.166975 2346 raft_consensus.cc:729] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871847ba83fc4d10ae5e1db23a4c7a62, State: Initialized, Role: FOLLOWER
I20260807 06:37:20.166976 2345 raft_consensus.cc:348] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.167095 2346 consensus_queue.cc:260] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.167172 2345 raft_consensus.cc:374] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:20.167209 2345 raft_consensus.cc:729] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4159f1c1eb3144d682a33e4b7296789a, State: Initialized, Role: FOLLOWER
I20260807 06:37:20.167344 2345 consensus_queue.cc:260] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.167382 2346 ts_tablet_manager.cc:1428] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 06:37:20.167542 1851 heartbeater.cc:503] Master 127.2.243.62:36517 was elected leader, sending a full tablet report...
I20260807 06:37:20.167629 2345 ts_tablet_manager.cc:1428] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:37:20.167645 2309 heartbeater.cc:503] Master 127.2.243.62:36517 was elected leader, sending a full tablet report...
I20260807 06:37:20.169065 2350 raft_consensus.cc:348] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.169248 2350 raft_consensus.cc:374] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:20.169312 2350 raft_consensus.cc:729] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1819e5cdb0472eb1a72e6edc98162c, State: Initialized, Role: FOLLOWER
I20260807 06:37:20.169421 2350 consensus_queue.cc:260] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.169732 2083 heartbeater.cc:503] Master 127.2.243.62:36517 was elected leader, sending a full tablet report...
I20260807 06:37:20.169937 2350 ts_tablet_manager.cc:1428] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260807 06:37:20.204319 2084 tablet.cc:2369] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:37:20.237077 2359 raft_consensus.cc:482] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:20.237239 2359 raft_consensus.cc:504] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.237635 2359 leader_election.cc:288] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 871847ba83fc4d10ae5e1db23a4c7a62 (127.2.243.1:36733), 4159f1c1eb3144d682a33e4b7296789a (127.2.243.3:38505)
I20260807 06:37:20.240777 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.237951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:41089 (local address 127.2.243.3:38505)
0807 06:37:20.237994 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.237996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.238005 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:20.238036 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.238037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.238050 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.238140 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.238325 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.238841 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.239984 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.240380 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.240469 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.240479 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.240669 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.240710 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:37:20.240784 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.237841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:38505 (local address 127.2.243.2:41089)
0807 06:37:20.237908 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.237920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.237965 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.238158 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.238159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.238161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.238236 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.238238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.238920 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.238921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.239871 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.239874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.240483 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.240484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.240596 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.240600 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.240658 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.240701 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":43}
I20260807 06:37:20.240789 2344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.237924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:48507 (local address 127.2.243.1:36733)
0807 06:37:20.237971 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.237973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.238147 (+ 174us) server_negotiation.cc:407] Connection header received
0807 06:37:20.238180 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.238181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.238193 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.238256 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.238426 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.238903 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.239970 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.240373 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.240429 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.240463 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.240664 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.240705 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:37:20.240794 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.237841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:36733 (local address 127.2.243.2:48507)
0807 06:37:20.238055 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.238067 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.238132 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.238263 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.238264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.238266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.238358 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.238359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.238985 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.238985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.239867 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.239870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.240464 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.240465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.240568 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.240578 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.240644 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.240689 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":26,"thread_start_us":119,"threads_started":1}
I20260807 06:37:20.241178 2234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479feee71bde49b1b0151bc39c3db8f8" candidate_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4159f1c1eb3144d682a33e4b7296789a" is_pre_election: true
I20260807 06:37:20.241343 1760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479feee71bde49b1b0151bc39c3db8f8" candidate_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" is_pre_election: true
I20260807 06:37:20.241456 2234 raft_consensus.cc:2454] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1819e5cdb0472eb1a72e6edc98162c in term 0.
I20260807 06:37:20.241542 1760 raft_consensus.cc:2454] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1819e5cdb0472eb1a72e6edc98162c in term 0.
I20260807 06:37:20.241707 1883 leader_election.cc:302] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4159f1c1eb3144d682a33e4b7296789a, 5d1819e5cdb0472eb1a72e6edc98162c; no voters:
I20260807 06:37:20.241858 2359 raft_consensus.cc:2781] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:37:20.241921 2359 raft_consensus.cc:482] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:37:20.241956 2359 raft_consensus.cc:3037] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:20.242699 2359 raft_consensus.cc:504] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.242848 2359 leader_election.cc:288] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [CANDIDATE]: Term 1 election: Requested vote from peers 871847ba83fc4d10ae5e1db23a4c7a62 (127.2.243.1:36733), 4159f1c1eb3144d682a33e4b7296789a (127.2.243.3:38505)
I20260807 06:37:20.242990 2234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479feee71bde49b1b0151bc39c3db8f8" candidate_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4159f1c1eb3144d682a33e4b7296789a"
I20260807 06:37:20.243000 1760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479feee71bde49b1b0151bc39c3db8f8" candidate_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871847ba83fc4d10ae5e1db23a4c7a62"
I20260807 06:37:20.243072 2234 raft_consensus.cc:3037] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:20.243093 1760 raft_consensus.cc:3037] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:20.243839 1760 raft_consensus.cc:2454] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1819e5cdb0472eb1a72e6edc98162c in term 1.
I20260807 06:37:20.243875 2234 raft_consensus.cc:2454] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1819e5cdb0472eb1a72e6edc98162c in term 1.
I20260807 06:37:20.244096 1885 leader_election.cc:302] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1819e5cdb0472eb1a72e6edc98162c, 871847ba83fc4d10ae5e1db23a4c7a62; no voters:
I20260807 06:37:20.244206 2359 raft_consensus.cc:2781] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:20.244374 2359 raft_consensus.cc:686] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 1 LEADER]: Becoming Leader. State: Replica: 5d1819e5cdb0472eb1a72e6edc98162c, State: Running, Role: LEADER
I20260807 06:37:20.244478 2359 consensus_queue.cc:237] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } }
I20260807 06:37:20.245340 1511 catalog_manager.cc:5495] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d1819e5cdb0472eb1a72e6edc98162c (127.2.243.2). New cstate: current_term: 1 leader_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d1819e5cdb0472eb1a72e6edc98162c" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 33989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 } health_report { overall_health: UNKNOWN } } }
I20260807 06:37:20.272544 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.270221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.2.243.2:33989)
0807 06:37:20.270286 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.270288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.270298 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:20.270340 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.270341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.270359 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.270418 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:20.270533 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.270981 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.271738 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.271921 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.271980 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.272207 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:20.272302 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:20.272365 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.272412 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.272427 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:37:20.274353 1760 raft_consensus.cc:1261] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 1 FOLLOWER]: Refusing update from remote peer 5d1819e5cdb0472eb1a72e6edc98162c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:20.274330 2234 raft_consensus.cc:1261] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 1 FOLLOWER]: Refusing update from remote peer 5d1819e5cdb0472eb1a72e6edc98162c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:20.274731 2361 consensus_queue.cc:1035] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [LEADER]: Connected to new peer: Peer: permanent_uuid: "871847ba83fc4d10ae5e1db23a4c7a62" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 36733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:20.274847 2359 consensus_queue.cc:1035] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4159f1c1eb3144d682a33e4b7296789a" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 38505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:20.277243 2367 mvcc.cc:205] Tried to move back new op lower bound from 7315802686561062912 to 7315802686442078208. Current Snapshot: MvccSnapshot[applied={T|T < 7315802686561062912}]
I20260807 06:37:20.277362 2369 mvcc.cc:205] Tried to move back new op lower bound from 7315802686561062912 to 7315802686442078208. Current Snapshot: MvccSnapshot[applied={T|T < 7315802686561062912}]
I20260807 06:37:20.278574 2370 mvcc.cc:205] Tried to move back new op lower bound from 7315802686561062912 to 7315802686442078208. Current Snapshot: MvccSnapshot[applied={T|T < 7315802686561062912 or (T in {7315802686561062912})}]
I20260807 06:37:20.279774 1511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:20.279873 1511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:20.280699 1511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c94511e94de4c4f9b3566b59519f1d0: Sending DeleteTablet for 3 replicas of tablet 479feee71bde49b1b0151bc39c3db8f8
I20260807 06:37:20.281000 1966 tablet_service.cc:1514] Processing DeleteTablet for tablet 479feee71bde49b1b0151bc39c3db8f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:20 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:50708
I20260807 06:37:20.281049 1733 tablet_service.cc:1514] Processing DeleteTablet for tablet 479feee71bde49b1b0151bc39c3db8f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:20 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:45906
I20260807 06:37:20.281244 2381 tablet_replica.cc:330] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: stopping tablet replica
I20260807 06:37:20.281267 2382 tablet_replica.cc:330] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62: stopping tablet replica
I20260807 06:37:20.281262 2195 tablet_service.cc:1514] Processing DeleteTablet for tablet 479feee71bde49b1b0151bc39c3db8f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:20 UTC) from {username='kudu', principal='kudu/127.2.243.62@KRBTEST.COM'} at 127.0.0.1:43714
I20260807 06:37:20.281356 2382 raft_consensus.cc:2229] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:20.281450 2382 raft_consensus.cc:2258] T 479feee71bde49b1b0151bc39c3db8f8 P 871847ba83fc4d10ae5e1db23a4c7a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:20.281471 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 1631
I20260807 06:37:20.281536 2383 tablet_replica.cc:330] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: stopping tablet replica
I20260807 06:37:20.281477 2381 raft_consensus.cc:2229] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:37:20.281646 2383 raft_consensus.cc:2229] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:20.281749 2383 raft_consensus.cc:2258] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:20.281745 2381 raft_consensus.cc:2258] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:20.282527 2381 ts_tablet_manager.cc:1905] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:20.284528 2381 ts_tablet_manager.cc:1918] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:20.284662 2381 log.cc:1196] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/479feee71bde49b1b0151bc39c3db8f8
I20260807 06:37:20.285020 2381 ts_tablet_manager.cc:1939] T 479feee71bde49b1b0151bc39c3db8f8 P 5d1819e5cdb0472eb1a72e6edc98162c: Deleting consensus metadata
I20260807 06:37:20.285059 2383 ts_tablet_manager.cc:1905] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:20.285454 1493 catalog_manager.cc:4841] TS 5d1819e5cdb0472eb1a72e6edc98162c (127.2.243.2:33989): tablet 479feee71bde49b1b0151bc39c3db8f8 (table test-table [id=1c21d3316dea4948b9276e8bfc1b23aa]) successfully deleted
I20260807 06:37:20.287108 2383 ts_tablet_manager.cc:1918] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:20.287204 2383 log.cc:1196] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/479feee71bde49b1b0151bc39c3db8f8
I20260807 06:37:20.287496 2383 ts_tablet_manager.cc:1939] T 479feee71bde49b1b0151bc39c3db8f8 P 4159f1c1eb3144d682a33e4b7296789a: Deleting consensus metadata
I20260807 06:37:20.287914 1487 catalog_manager.cc:4841] TS 4159f1c1eb3144d682a33e4b7296789a (127.2.243.3:38505): tablet 479feee71bde49b1b0151bc39c3db8f8 (table test-table [id=1c21d3316dea4948b9276e8bfc1b23aa]) successfully deleted
W20260807 06:37:20.294379 1493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:36733 (error 108)
I20260807 06:37:20.294958 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.294808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:36733 (local address 127.0.0.1:45912)
0807 06:37:20.294872 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.294924 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:36733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:37:20.295048 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 1856
W20260807 06:37:20.299397 1493 catalog_manager.cc:4584] TS 871847ba83fc4d10ae5e1db23a4c7a62 (127.2.243.1:36733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 479feee71bde49b1b0151bc39c3db8f8: Network error: Client connection negotiation failed: client connection to 127.2.243.1:36733: connect: Connection refused (error 111)
I20260807 06:37:20.306931 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 2089
I20260807 06:37:20.318425 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 1461
2026-08-07T06:37:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2218 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T06:37:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:20Z Disabled control of system clock
I20260807 06:37:20.355780 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39433
--webserver_interface=127.2.243.62
--webserver_port=0
--builtin_ntp_servers=127.2.243.20:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:39433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:20.436942 2389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:20.437152 2389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:20.437172 2389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:20.438249 2389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:20.438284 2389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:20.438297 2389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:20.438309 2389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:20.438321 2389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:20.439673 2389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:39433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:39433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:20.439862 2389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:20.440080 2389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:20.444746 2400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.444757 2403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.444757 2398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:20.444773 2389 server_base.cc:1034] running on GCE node
I20260807 06:37:20.445219 2389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:20.445528 2389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:20.446672 2389 hybrid_clock.cc:648] HybridClock initialized: now 1786084640446660 us; error 29 us; skew 500 ppm
I20260807 06:37:20.447778 2389 webserver.cc:460] Webserver started at http://127.2.243.62:44139/ using document root <none> and password file <none>
I20260807 06:37:20.447985 2389 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:20.448026 2389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:20.448163 2389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:20.449208 2389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "3fee08c18bbc49df99d95a295577a3f4"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "5fe50c368e03ed9c2511f37246d5cb8a"
server_key_iv: "c821c20f561901df5f3bc91952d0f0bb"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.449515 2389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "3fee08c18bbc49df99d95a295577a3f4"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "5fe50c368e03ed9c2511f37246d5cb8a"
server_key_iv: "c821c20f561901df5f3bc91952d0f0bb"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.450832 2389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:20.451606 2412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.451844 2389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:20.451956 2389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "3fee08c18bbc49df99d95a295577a3f4"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "5fe50c368e03ed9c2511f37246d5cb8a"
server_key_iv: "c821c20f561901df5f3bc91952d0f0bb"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.452055 2389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:20.463096 2389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:20.463490 2389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:20.463665 2389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:20.468040 2389 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:39433
I20260807 06:37:20.468065 2504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:39433 every 8 connection(s)
I20260807 06:37:20.468442 2389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:20.469489 2507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:20.470075 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 2389
I20260807 06:37:20.470172 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:20.470443 3020 external_mini_cluster.cc:1428] Setting key 75cf261ca429c7b60f3bd9586cffe1a0
I20260807 06:37:20.472597 2507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4: Bootstrap starting.
I20260807 06:37:20.473270 2507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:20.473583 2507 log.cc:824] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:20.474366 2507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4: No bootstrap required, opened a new log
I20260807 06:37:20.476220 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.471056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.2.243.62:39433)
0807 06:37:20.471239 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.471246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.471270 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:37:20.471671 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.471678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.471733 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.471842 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:20.472115 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.473086 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.474131 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.474606 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.474757 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.475576 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:20.475589 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:20.475589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:20.475759 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:20.475792 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:20.475803 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:20.475881 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.475962 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.476061 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260807 06:37:20.476576 2507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fee08c18bbc49df99d95a295577a3f4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39433 } }
I20260807 06:37:20.476759 2507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:20.476794 2507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fee08c18bbc49df99d95a295577a3f4, State: Initialized, Role: FOLLOWER
I20260807 06:37:20.476907 2507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fee08c18bbc49df99d95a295577a3f4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39433 } }
I20260807 06:37:20.476997 2507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:20.477029 2507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:20.477084 2507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:20.477629 2507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fee08c18bbc49df99d95a295577a3f4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39433 } }
I20260807 06:37:20.477743 2507 leader_election.cc:302] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fee08c18bbc49df99d95a295577a3f4; no voters:
I20260807 06:37:20.477937 2507 leader_election.cc:288] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:20.477986 2515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:20.478176 2515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [term 1 LEADER]: Becoming Leader. State: Replica: 3fee08c18bbc49df99d95a295577a3f4, State: Running, Role: LEADER
I20260807 06:37:20.478243 2507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:20.478291 2515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fee08c18bbc49df99d95a295577a3f4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39433 } }
I20260807 06:37:20.478724 2515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fee08c18bbc49df99d95a295577a3f4. Latest consensus state: current_term: 1 leader_uuid: "3fee08c18bbc49df99d95a295577a3f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fee08c18bbc49df99d95a295577a3f4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39433 } } }
I20260807 06:37:20.478736 2519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fee08c18bbc49df99d95a295577a3f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fee08c18bbc49df99d95a295577a3f4" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 39433 } } }
I20260807 06:37:20.478820 2515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:20.478822 2519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:20.479478 2526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:20.480005 2526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:20.481603 2526 catalog_manager.cc:1347] Generated new cluster ID: 62aa9e72e0d94e3a9d8c66a3d1e0519c
I20260807 06:37:20.481658 2526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:20.486905 2526 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:20.487427 2526 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:20.490170 2526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4: Generated new TSK 0
I20260807 06:37:20.490334 2526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:20.493954 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.1:0
--local_ip_for_outbound_sockets=127.2.243.1
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39433
--builtin_ntp_servers=127.2.243.20:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:20.578723 2544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:20.578938 2544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:20.578974 2544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:20.580348 2544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:20.580413 2544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:20.580454 2544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.1
I20260807 06:37:20.581852 2544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.243.1
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.1
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:20.582124 2544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:20.582435 2544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:20.588284 2556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.588274 2551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:20.588543 2544 server_base.cc:1034] running on GCE node
W20260807 06:37:20.588284 2558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:20.588867 2544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:20.589171 2544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:20.590384 2544 hybrid_clock.cc:648] HybridClock initialized: now 1786084640590371 us; error 36 us; skew 500 ppm
I20260807 06:37:20.591907 2544 webserver.cc:460] Webserver started at http://127.2.243.1:38287/ using document root <none> and password file <none>
I20260807 06:37:20.592176 2544 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:20.592233 2544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:20.592348 2544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:20.593660 2544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data/instance:
uuid: "6de80afb3eb14fc487a501dd1cfdddbf"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "8b282241c2b9542ab8c762737974cc73"
server_key_iv: "670afb10374f1415ac456eabe2a17358"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.594031 2544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance:
uuid: "6de80afb3eb14fc487a501dd1cfdddbf"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "8b282241c2b9542ab8c762737974cc73"
server_key_iv: "670afb10374f1415ac456eabe2a17358"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.595546 2544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:20.596446 2567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.596657 2544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:20.596748 2544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/wal
uuid: "6de80afb3eb14fc487a501dd1cfdddbf"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "8b282241c2b9542ab8c762737974cc73"
server_key_iv: "670afb10374f1415ac456eabe2a17358"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.596817 2544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:20.609627 2544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:20.609939 2544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:20.610090 2544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:20.610312 2544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:20.610649 2544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:20.610705 2544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.610749 2544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:20.610776 2544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.616278 2544 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.1:38139
I20260807 06:37:20.616356 2758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.1:38139 every 8 connection(s)
I20260807 06:37:20.616715 2544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/data/info.pb
I20260807 06:37:20.619864 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 2544
I20260807 06:37:20.619951 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-0/wal/instance
I20260807 06:37:20.620167 3020 external_mini_cluster.cc:1428] Setting key a102086be8937e0092ed4859535ee659
I20260807 06:37:20.621212 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.2:0
--local_ip_for_outbound_sockets=127.2.243.2
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39433
--builtin_ntp_servers=127.2.243.20:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:37:20.622013 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.617460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:33979 (local address 127.2.243.62:39433)
0807 06:37:20.617519 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.617522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.617778 (+ 256us) server_negotiation.cc:407] Connection header received
0807 06:37:20.618243 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.618244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.618256 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.618311 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:20.618921 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.619408 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.620514 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.620740 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.620820 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.621771 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:20.621772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:20.621772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:20.621776 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:20.621792 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:20.621796 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:20.621844 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.621934 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.621957 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 06:37:20.622289 2764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.617462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39433 (local address 127.2.243.1:33979)
0807 06:37:20.617658 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.617691 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.618163 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.618361 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.618364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.618372 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:20.618853 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.618857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.619485 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.619486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.620397 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.620400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.620842 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.620843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.620995 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.621611 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:20.621612 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:20.621648 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:20.621678 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:20.621860 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:20.621864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:20.621867 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.621930 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.622045 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260807 06:37:20.622814 2761 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39433
I20260807 06:37:20.622922 2761 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:20.623109 2761 heartbeater.cc:511] Master 127.2.243.62:39433 requested a full tablet report, sending...
I20260807 06:37:20.623617 2440 ts_manager.cc:194] Registered new tserver with Master: 6de80afb3eb14fc487a501dd1cfdddbf (127.2.243.1:38139)
I20260807 06:37:20.624202 2440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.1:33979
W20260807 06:37:20.703615 2765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:20.703800 2765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:20.703820 2765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:20.704973 2765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:20.705014 2765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:20.705039 2765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.2
I20260807 06:37:20.706360 2765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.243.2
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.2
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:20.706581 2765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:20.706875 2765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:20.711901 2778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.711901 2775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.712002 2776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:20.712411 2765 server_base.cc:1034] running on GCE node
I20260807 06:37:20.712641 2765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:20.712886 2765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:20.714076 2765 hybrid_clock.cc:648] HybridClock initialized: now 1786084640714055 us; error 40 us; skew 500 ppm
I20260807 06:37:20.715541 2765 webserver.cc:460] Webserver started at http://127.2.243.2:40475/ using document root <none> and password file <none>
I20260807 06:37:20.715809 2765 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:20.715881 2765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:20.716037 2765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:20.717156 2765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data/instance:
uuid: "c203eb87b9204f09bcf89f53237bf0fd"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "75f29ba3f89ee05e9d3916f1ec62a2cc"
server_key_iv: "8a9a2f55fc2d26f7fdbb2952dce23631"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.717538 2765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance:
uuid: "c203eb87b9204f09bcf89f53237bf0fd"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "75f29ba3f89ee05e9d3916f1ec62a2cc"
server_key_iv: "8a9a2f55fc2d26f7fdbb2952dce23631"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.718828 2765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:20.719651 2793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.719893 2765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:20.719995 2765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal
uuid: "c203eb87b9204f09bcf89f53237bf0fd"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "75f29ba3f89ee05e9d3916f1ec62a2cc"
server_key_iv: "8a9a2f55fc2d26f7fdbb2952dce23631"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.720088 2765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:20.751312 2765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:20.751674 2765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:20.751861 2765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:20.752170 2765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:20.752599 2765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:20.752658 2765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.752701 2765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:20.752772 2765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.759511 2765 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.2:39791
I20260807 06:37:20.759579 2985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.2:39791 every 8 connection(s)
I20260807 06:37:20.759927 2765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/data/info.pb
I20260807 06:37:20.765522 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.760677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.2:59581 (local address 127.2.243.62:39433)
0807 06:37:20.760748 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.760750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.760989 (+ 239us) server_negotiation.cc:407] Connection header received
0807 06:37:20.761508 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.761509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.761523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.761589 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:20.762300 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.762799 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.763856 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.764069 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.764168 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.765199 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:20.765200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:20.765200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:20.765205 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:20.765221 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:20.765226 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:20.765287 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.765392 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.765421 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:37:20.765723 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.760663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39433 (local address 127.2.243.2:59581)
0807 06:37:20.760863 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.760895 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.761399 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.761656 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.761660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.761669 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:20.762185 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.762190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.762905 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.762906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.763739 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.763742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.764180 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.764181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.764313 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.765029 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:20.765029 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:20.765059 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:20.765082 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:20.765293 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:20.765298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:20.765301 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.765394 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.765497 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260807 06:37:20.766317 2988 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39433
I20260807 06:37:20.766402 2988 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:20.766593 2988 heartbeater.cc:511] Master 127.2.243.62:39433 requested a full tablet report, sending...
I20260807 06:37:20.767036 2440 ts_manager.cc:194] Registered new tserver with Master: c203eb87b9204f09bcf89f53237bf0fd (127.2.243.2:39791)
I20260807 06:37:20.767377 2440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.2:59581
I20260807 06:37:20.767907 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 2765
I20260807 06:37:20.768018 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal/instance
I20260807 06:37:20.768229 3020 external_mini_cluster.cc:1428] Setting key 5fd8b189d2b4ca74b7133cdbc64888e6
I20260807 06:37:20.769183 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.243.3:0
--local_ip_for_outbound_sockets=127.2.243.3
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39433
--builtin_ntp_servers=127.2.243.20:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:37:20.851349 2994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:20.851584 2994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:20.851621 2994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:20.852826 2994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:20.852885 2994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:20.852929 2994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.243.3
I20260807 06:37:20.854322 2994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.243.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.243.3
--webserver_port=0
--tserver_master_addrs=127.2.243.62:39433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.243.3
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:20.854593 2994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:20.854975 2994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:20.860054 3002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:20.860193 3003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:20.860258 2994 server_base.cc:1034] running on GCE node
W20260807 06:37:20.860167 3005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:20.860598 2994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:20.860874 2994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:20.862056 2994 hybrid_clock.cc:648] HybridClock initialized: now 1786084640862039 us; error 48 us; skew 500 ppm
I20260807 06:37:20.863417 2994 webserver.cc:460] Webserver started at http://127.2.243.3:39273/ using document root <none> and password file <none>
I20260807 06:37:20.863686 2994 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:20.863759 2994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:20.863931 2994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:20.865209 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data/instance:
uuid: "658295333b6c430bab882cc2d2c32e57"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "dfff3b8810f2a0aaf5999ca8dadd079a"
server_key_iv: "40ddaa7f4e22e8f1b5a85bf6c5a56de7"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.865587 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance:
uuid: "658295333b6c430bab882cc2d2c32e57"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "dfff3b8810f2a0aaf5999ca8dadd079a"
server_key_iv: "40ddaa7f4e22e8f1b5a85bf6c5a56de7"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.866927 2994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:20.867770 3013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.867990 2994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:20.868098 2994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal
uuid: "658295333b6c430bab882cc2d2c32e57"
format_stamp: "Formatted at 2026-08-07 06:37:20 on dist-test-slave-rwrg"
server_key: "dfff3b8810f2a0aaf5999ca8dadd079a"
server_key_iv: "40ddaa7f4e22e8f1b5a85bf6c5a56de7"
server_key_version: "encryptionkey@0"
I20260807 06:37:20.868183 2994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:20.906483 2994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:20.906833 2994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:20.907003 2994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:20.907250 2994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:37:20.907624 2994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:37:20.907681 2994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.907717 2994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:37:20.907744 2994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:20.913563 2994 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.3:37903
I20260807 06:37:20.913612 3214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.3:37903 every 8 connection(s)
I20260807 06:37:20.913967 2994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/data/info.pb
I20260807 06:37:20.915004 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 2994
I20260807 06:37:20.915130 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal/instance
I20260807 06:37:20.915393 3020 external_mini_cluster.cc:1428] Setting key f5d511a23ad88a80dfb3b682f0f72db0
I20260807 06:37:20.920025 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.914882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.3:34439 (local address 127.2.243.62:39433)
0807 06:37:20.914971 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.914975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.915302 (+ 327us) server_negotiation.cc:407] Connection header received
0807 06:37:20.916143 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.916144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.916156 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.916215 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:20.917069 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.917551 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.918686 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.918881 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.918945 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.919794 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:20.919795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:20.919796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:20.919799 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:20.919811 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:20.919814 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:20.919856 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.919939 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.919958 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:37:20.920316 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.914852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:39433 (local address 127.2.243.3:34439)
0807 06:37:20.915126 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.915202 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.915993 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.916297 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.916301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.916312 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:20.916972 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.916986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.917675 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.917676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.918587 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.918590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.918989 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.918990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.919117 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.919655 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:37:20.919656 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:20.919687 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:37:20.919711 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:20.919870 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:20.919872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:20.919875 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.919932 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.920034 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260807 06:37:20.920985 3217 heartbeater.cc:347] Connected to a master server at 127.2.243.62:39433
I20260807 06:37:20.921121 3217 heartbeater.cc:464] Registering TS with master...
I20260807 06:37:20.921324 3217 heartbeater.cc:511] Master 127.2.243.62:39433 requested a full tablet report, sending...
I20260807 06:37:20.921793 2440 ts_manager.cc:194] Registered new tserver with Master: 658295333b6c430bab882cc2d2c32e57 (127.2.243.3:37903)
I20260807 06:37:20.922230 2440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.243.3:34439
I20260807 06:37:20.927246 3020 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:37:20.931651 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.928704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.2.243.62:39433)
0807 06:37:20.928780 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.928782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.928891 (+ 109us) server_negotiation.cc:407] Connection header received
0807 06:37:20.928969 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.928970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.928979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.929051 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:20.929249 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.929757 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.930529 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.930727 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.930803 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.931414 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:20.931416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:20.931416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:20.931421 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:20.931435 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:20.931439 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:20.931491 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.931551 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.931570 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:37:20.933284 2440 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:37:20.934859 2440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:37:20.941452 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.937215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:38139 (local address 127.0.0.1:59130)
0807 06:37:20.937853 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.937862 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.937917 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.938217 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.938218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.938221 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.938262 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.938263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.939168 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.939169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.940213 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.940216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.941145 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.941146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.941223 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.941238 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.941321 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.941371 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260807 06:37:20.941628 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.937253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.2.243.2:39791)
0807 06:37:20.937439 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.937443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.937478 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:37:20.937517 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.937519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.937541 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.937637 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.937802 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.938926 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.940443 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.941146 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.941206 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.941299 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.941500 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.941558 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":83,"threads_started":1}
I20260807 06:37:20.941643 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.937420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.2.243.3:37903)
0807 06:37:20.937582 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.937587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.937798 (+ 211us) server_negotiation.cc:407] Connection header received
0807 06:37:20.938096 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.938099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.938113 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.938183 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.938359 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.939461 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.940701 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.941232 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.941328 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.941406 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.941504 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.941556 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":101,"threads_started":1}
I20260807 06:37:20.941839 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.937619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.2.243.1:38139)
0807 06:37:20.937782 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:20.937786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:20.937950 (+ 164us) server_negotiation.cc:407] Connection header received
0807 06:37:20.938022 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:20.938025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:20.938051 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:20.938143 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:20.938344 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.939077 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.940281 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.941051 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.941132 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.941226 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:20.941717 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 06:37:20.941762 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260807 06:37:20.941577 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.937166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:39791 (local address 127.0.0.1:46550)
0807 06:37:20.937332 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.937369 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.937434 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.937642 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.937645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.937647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.937708 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.937710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.939027 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.939028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.940352 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.940355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.941338 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.941339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.941404 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.941412 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.941471 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.941510 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":61,"thread_start_us":64,"threads_started":1}
I20260807 06:37:20.941860 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:20.937215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:37903 (local address 127.0.0.1:53438)
0807 06:37:20.937731 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:20.937741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:20.937969 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:20.938237 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:20.938238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:20.938240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:20.938267 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.938268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.939546 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.939547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.940624 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:20.940626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:20.941340 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:20.941340 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:20.941404 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:20.941411 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:37:20.941758 (+ 347us) client_negotiation.cc:240] Negotiation successful
0807 06:37:20.941808 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260807 06:37:20.942795 3098 tablet_service.cc:1467] Processing CreateTablet for tablet d90cbfa554c44f2fa8807dcff52d8c2a (DEFAULT_TABLE table=test-table [id=aa3d04da7335450b869bedc4b555a88d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:20.942744 2872 tablet_service.cc:1467] Processing CreateTablet for tablet d90cbfa554c44f2fa8807dcff52d8c2a (DEFAULT_TABLE table=test-table [id=aa3d04da7335450b869bedc4b555a88d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:20.942746 2635 tablet_service.cc:1467] Processing CreateTablet for tablet d90cbfa554c44f2fa8807dcff52d8c2a (DEFAULT_TABLE table=test-table [id=aa3d04da7335450b869bedc4b555a88d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:37:20.943102 2872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d90cbfa554c44f2fa8807dcff52d8c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:20.943102 2635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d90cbfa554c44f2fa8807dcff52d8c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:20.943143 3098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d90cbfa554c44f2fa8807dcff52d8c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:20.945946 3239 tablet_bootstrap.cc:492] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Bootstrap starting.
I20260807 06:37:20.946341 3241 tablet_bootstrap.cc:492] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Bootstrap starting.
I20260807 06:37:20.946659 3239 tablet_bootstrap.cc:654] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:20.946993 3241 tablet_bootstrap.cc:654] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:20.946992 3239 log.cc:824] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:20.947367 3241 log.cc:824] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:20.947726 3240 tablet_bootstrap.cc:492] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf: Bootstrap starting.
I20260807 06:37:20.947827 3239 tablet_bootstrap.cc:492] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: No bootstrap required, opened a new log
I20260807 06:37:20.947896 3239 ts_tablet_manager.cc:1397] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:37:20.948377 3240 tablet_bootstrap.cc:654] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:20.948764 3240 log.cc:824] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:20.948894 3241 tablet_bootstrap.cc:492] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: No bootstrap required, opened a new log
I20260807 06:37:20.948974 3241 ts_tablet_manager.cc:1397] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 06:37:20.949610 3240 tablet_bootstrap.cc:492] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf: No bootstrap required, opened a new log
I20260807 06:37:20.949698 3240 ts_tablet_manager.cc:1397] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:37:20.950304 3239 raft_consensus.cc:348] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:20.950460 3239 raft_consensus.cc:374] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:20.950505 3239 raft_consensus.cc:729] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658295333b6c430bab882cc2d2c32e57, State: Initialized, Role: FOLLOWER
I20260807 06:37:20.950627 3239 consensus_queue.cc:260] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:20.950919 3239 ts_tablet_manager.cc:1428] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:37:20.950945 3217 heartbeater.cc:503] Master 127.2.243.62:39433 was elected leader, sending a full tablet report...
I20260807 06:37:20.951633 3241 raft_consensus.cc:348] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:20.951792 3241 raft_consensus.cc:374] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:20.951828 3241 raft_consensus.cc:729] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c203eb87b9204f09bcf89f53237bf0fd, State: Initialized, Role: FOLLOWER
I20260807 06:37:20.951961 3241 consensus_queue.cc:260] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:20.952198 3241 ts_tablet_manager.cc:1428] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 06:37:20.952270 2988 heartbeater.cc:503] Master 127.2.243.62:39433 was elected leader, sending a full tablet report...
I20260807 06:37:20.952201 3240 raft_consensus.cc:348] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:20.952334 3240 raft_consensus.cc:374] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:20.952378 3240 raft_consensus.cc:729] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de80afb3eb14fc487a501dd1cfdddbf, State: Initialized, Role: FOLLOWER
I20260807 06:37:20.952486 3240 consensus_queue.cc:260] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:20.952798 3240 ts_tablet_manager.cc:1428] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 06:37:20.952880 2761 heartbeater.cc:503] Master 127.2.243.62:39433 was elected leader, sending a full tablet report...
W20260807 06:37:21.010527 2989 tablet.cc:2369] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:37:21.117424 2762 tablet.cc:2369] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:37:21.164619 3218 tablet.cc:2369] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:37:21.230417 3252 raft_consensus.cc:482] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:37:21.230595 3252 raft_consensus.cc:504] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:21.231035 3252 leader_election.cc:288] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c203eb87b9204f09bcf89f53237bf0fd (127.2.243.2:39791), 658295333b6c430bab882cc2d2c32e57 (127.2.243.3:37903)
I20260807 06:37:21.235394 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:21.231412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:35295 (local address 127.2.243.2:39791)
0807 06:37:21.231452 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:21.231453 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:21.231573 (+ 120us) server_negotiation.cc:407] Connection header received
0807 06:37:21.231604 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:21.231605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:21.231618 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:21.231688 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:21.231955 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.232795 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.234439 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.234998 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.235076 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:21.235086 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:21.235284 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 06:37:21.235322 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:37:21.235394 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:21.231373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.243.1:40981 (local address 127.2.243.3:37903)
0807 06:37:21.231426 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:21.231428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:21.231531 (+ 103us) server_negotiation.cc:407] Connection header received
0807 06:37:21.231605 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:21.231605 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:21.231616 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:21.231688 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:37:21.231955 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.232761 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.234439 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.234998 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.235076 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:21.235086 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:21.235284 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 06:37:21.235322 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:37:21.235471 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:21.231300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.2:39791 (local address 127.2.243.1:35295)
0807 06:37:21.231441 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:21.231454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:21.231530 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:21.231715 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:21.231716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:21.231718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:21.231831 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:21.231836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.232887 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.232888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:21.234327 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:21.234328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.235111 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.235112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:21.235193 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:21.235199 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:37:21.235270 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:37:21.235325 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":88,"thread_start_us":70,"threads_started":1}
I20260807 06:37:21.235474 3257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:21.231300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.3:37903 (local address 127.2.243.1:40981)
0807 06:37:21.231441 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:21.231454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:21.231530 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:21.231715 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:21.231716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:21.231719 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:37:21.231831 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:21.231834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.232887 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.232888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:21.234324 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:21.234327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.235108 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.235109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:21.235204 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:21.235206 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 06:37:21.235270 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:37:21.235324 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":129,"thread_start_us":68,"threads_started":1}
I20260807 06:37:21.235922 2907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90cbfa554c44f2fa8807dcff52d8c2a" candidate_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c203eb87b9204f09bcf89f53237bf0fd" is_pre_election: true
I20260807 06:37:21.236107 2907 raft_consensus.cc:2454] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de80afb3eb14fc487a501dd1cfdddbf in term 0.
I20260807 06:37:21.236212 3135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90cbfa554c44f2fa8807dcff52d8c2a" candidate_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658295333b6c430bab882cc2d2c32e57" is_pre_election: true
I20260807 06:37:21.236398 3135 raft_consensus.cc:2454] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de80afb3eb14fc487a501dd1cfdddbf in term 0.
I20260807 06:37:21.236388 2569 leader_election.cc:302] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de80afb3eb14fc487a501dd1cfdddbf, c203eb87b9204f09bcf89f53237bf0fd; no voters:
I20260807 06:37:21.236541 3252 raft_consensus.cc:2781] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:37:21.236617 3252 raft_consensus.cc:482] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:37:21.236662 3252 raft_consensus.cc:3037] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:21.237377 3252 raft_consensus.cc:504] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:21.237528 3252 leader_election.cc:288] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [CANDIDATE]: Term 1 election: Requested vote from peers c203eb87b9204f09bcf89f53237bf0fd (127.2.243.2:39791), 658295333b6c430bab882cc2d2c32e57 (127.2.243.3:37903)
I20260807 06:37:21.237673 2907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90cbfa554c44f2fa8807dcff52d8c2a" candidate_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c203eb87b9204f09bcf89f53237bf0fd"
I20260807 06:37:21.237694 3135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90cbfa554c44f2fa8807dcff52d8c2a" candidate_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658295333b6c430bab882cc2d2c32e57"
I20260807 06:37:21.237766 3135 raft_consensus.cc:3037] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:21.237790 2907 raft_consensus.cc:3037] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:21.238432 3135 raft_consensus.cc:2454] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de80afb3eb14fc487a501dd1cfdddbf in term 1.
I20260807 06:37:21.238621 2571 leader_election.cc:302] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658295333b6c430bab882cc2d2c32e57, 6de80afb3eb14fc487a501dd1cfdddbf; no voters:
I20260807 06:37:21.238696 2907 raft_consensus.cc:2454] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de80afb3eb14fc487a501dd1cfdddbf in term 1.
I20260807 06:37:21.238763 3252 raft_consensus.cc:2781] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:21.239048 3252 raft_consensus.cc:686] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [term 1 LEADER]: Becoming Leader. State: Replica: 6de80afb3eb14fc487a501dd1cfdddbf, State: Running, Role: LEADER
I20260807 06:37:21.239148 3252 consensus_queue.cc:237] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } }
I20260807 06:37:21.239985 2440 catalog_manager.cc:5495] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 6de80afb3eb14fc487a501dd1cfdddbf (127.2.243.1). New cstate: current_term: 1 leader_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de80afb3eb14fc487a501dd1cfdddbf" member_type: VOTER last_known_addr { host: "127.2.243.1" port: 38139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 } health_report { overall_health: UNKNOWN } } }
I20260807 06:37:21.293979 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:21.291711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.2.243.1:38139)
0807 06:37:21.291773 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:21.291775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:21.291783 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:37:21.291822 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:21.291823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:21.291842 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:21.291904 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:21.292026 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.292482 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.293250 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:21.293430 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:21.293481 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:21.293656 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:21.293757 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:21.293819 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:21.293865 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 06:37:21.293882 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 06:37:21.295889 3135 raft_consensus.cc:1261] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 1 FOLLOWER]: Refusing update from remote peer 6de80afb3eb14fc487a501dd1cfdddbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:21.296196 2907 raft_consensus.cc:1261] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 1 FOLLOWER]: Refusing update from remote peer 6de80afb3eb14fc487a501dd1cfdddbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:21.296358 3259 consensus_queue.cc:1035] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "658295333b6c430bab882cc2d2c32e57" member_type: VOTER last_known_addr { host: "127.2.243.3" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:21.296618 3259 consensus_queue.cc:1035] T d90cbfa554c44f2fa8807dcff52d8c2a P 6de80afb3eb14fc487a501dd1cfdddbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "c203eb87b9204f09bcf89f53237bf0fd" member_type: VOTER last_known_addr { host: "127.2.243.2" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:21.299003 3266 mvcc.cc:205] Tried to move back new op lower bound from 7315802690744934400 to 7315802690516176896. Current Snapshot: MvccSnapshot[applied={T|T < 7315802690744934400}]
I20260807 06:37:21.299470 3268 mvcc.cc:205] Tried to move back new op lower bound from 7315802690744934400 to 7315802690516176896. Current Snapshot: MvccSnapshot[applied={T|T < 7315802690744934400}]
I20260807 06:37:21.299518 3265 mvcc.cc:205] Tried to move back new op lower bound from 7315802690744934400 to 7315802690516176896. Current Snapshot: MvccSnapshot[applied={T|T < 7315802690744934400 or (T in {7315802690744934400})}]
I20260807 06:37:21.301057 2439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:21.301158 2439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:37:21.301955 2439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fee08c18bbc49df99d95a295577a3f4: Sending DeleteTablet for 3 replicas of tablet d90cbfa554c44f2fa8807dcff52d8c2a
I20260807 06:37:21.302309 2635 tablet_service.cc:1514] Processing DeleteTablet for tablet d90cbfa554c44f2fa8807dcff52d8c2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:21 UTC) from {username='slave'} at 127.0.0.1:59130
I20260807 06:37:21.302316 2872 tablet_service.cc:1514] Processing DeleteTablet for tablet d90cbfa554c44f2fa8807dcff52d8c2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:21 UTC) from {username='slave'} at 127.0.0.1:46550
I20260807 06:37:21.302340 3098 tablet_service.cc:1514] Processing DeleteTablet for tablet d90cbfa554c44f2fa8807dcff52d8c2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:37:21 UTC) from {username='slave'} at 127.0.0.1:53438
I20260807 06:37:21.302631 3282 tablet_replica.cc:330] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: stopping tablet replica
I20260807 06:37:21.302733 3282 raft_consensus.cc:2229] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:21.302848 3282 raft_consensus.cc:2258] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:21.302909 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 2544
I20260807 06:37:21.303309 3282 ts_tablet_manager.cc:1905] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:21.303452 3281 tablet_replica.cc:330] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: stopping tablet replica
I20260807 06:37:21.303546 3281 raft_consensus.cc:2229] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:37:21.303635 3281 raft_consensus.cc:2258] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:37:21.305724 3282 ts_tablet_manager.cc:1918] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:21.305804 3282 log.cc:1196] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-1/wal/wals/d90cbfa554c44f2fa8807dcff52d8c2a
I20260807 06:37:21.306098 3282 ts_tablet_manager.cc:1939] T d90cbfa554c44f2fa8807dcff52d8c2a P c203eb87b9204f09bcf89f53237bf0fd: Deleting consensus metadata
I20260807 06:37:21.306089 3281 ts_tablet_manager.cc:1905] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:37:21.306557 2415 catalog_manager.cc:4841] TS c203eb87b9204f09bcf89f53237bf0fd (127.2.243.2:39791): tablet d90cbfa554c44f2fa8807dcff52d8c2a (table test-table [id=aa3d04da7335450b869bedc4b555a88d]) successfully deleted
I20260807 06:37:21.308888 3281 ts_tablet_manager.cc:1918] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:37:21.309031 3281 log.cc:1196] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Deleting WAL directory at /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786084392481039-3020-0/minicluster-data/ts-2/wal/wals/d90cbfa554c44f2fa8807dcff52d8c2a
I20260807 06:37:21.309423 3281 ts_tablet_manager.cc:1939] T d90cbfa554c44f2fa8807dcff52d8c2a P 658295333b6c430bab882cc2d2c32e57: Deleting consensus metadata
I20260807 06:37:21.309894 2416 catalog_manager.cc:4841] TS 658295333b6c430bab882cc2d2c32e57 (127.2.243.3:37903): tablet d90cbfa554c44f2fa8807dcff52d8c2a (table test-table [id=aa3d04da7335450b869bedc4b555a88d]) successfully deleted
W20260807 06:37:21.319132 2418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.243.1:38139 (error 108)
I20260807 06:37:21.319336 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 2765
I20260807 06:37:21.319695 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:21.319564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.1:38139 (local address 127.0.0.1:59148)
0807 06:37:21.319619 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:21.319656 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.1:38139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:37:21.319929 2418 catalog_manager.cc:4584] TS 6de80afb3eb14fc487a501dd1cfdddbf (127.2.243.1:38139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d90cbfa554c44f2fa8807dcff52d8c2a: Network error: Client connection negotiation failed: client connection to 127.2.243.1:38139: connect: Connection refused (error 111)
I20260807 06:37:21.331264 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 2994
I20260807 06:37:21.339504 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 2389
2026-08-07T06:37:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1017 ms)
[----------] 6 tests from EncryptionPolicyTest (9591 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:21 dist-test-slave-rwrg krb5kdc[3288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:21 dist-test-slave-rwrg krb5kdc[3288](info): set up 2 sockets
Aug 07 06:37:21 dist-test-slave-rwrg krb5kdc[3288](info): commencing operation
krb5kdc: starting...
W20260807 06:37:22.729837 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:22 dist-test-slave-rwrg krb5kdc[3288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:22.853852 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41217
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:22.936473 3314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:22.936718 3314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:22.936740 3314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:22.937930 3314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:22.937973 3314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:22.937988 3314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:22.938000 3314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:22.938012 3314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:22.939502 3314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41217
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:22.939736 3314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:22.939977 3314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:22.945026 3325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:22.945027 3327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:22.945047 3323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:22.945133 3314 server_base.cc:1034] running on GCE node
I20260807 06:37:22.945485 3314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:22.945818 3314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:22.947024 3314 hybrid_clock.cc:648] HybridClock initialized: now 1786084642947015 us; error 35 us; skew 500 ppm
Aug 07 06:37:22 dist-test-slave-rwrg krb5kdc[3288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084642, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:22.949745 3314 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:22.950196 3314 webserver.cc:460] Webserver started at http://127.0.0.1:36479/ using document root <none> and password file <none>
I20260807 06:37:22.950454 3314 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:22.950523 3314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:22.950673 3314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:22.951730 3314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "cec7412f3de04fd3b540341042553a57"
format_stamp: "Formatted at 2026-08-07 06:37:22 on dist-test-slave-rwrg"
server_key: "45ab27ed9e701593af0a810db731143c"
server_key_iv: "2dcc0b588325a2bcaf159300fff410c1"
server_key_version: "encryptionkey@0"
I20260807 06:37:22.952090 3314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "cec7412f3de04fd3b540341042553a57"
format_stamp: "Formatted at 2026-08-07 06:37:22 on dist-test-slave-rwrg"
server_key: "45ab27ed9e701593af0a810db731143c"
server_key_iv: "2dcc0b588325a2bcaf159300fff410c1"
server_key_version: "encryptionkey@0"
I20260807 06:37:22.953539 3314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:22.954355 3339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:22.954589 3314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:37:22.954694 3314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "cec7412f3de04fd3b540341042553a57"
format_stamp: "Formatted at 2026-08-07 06:37:22 on dist-test-slave-rwrg"
server_key: "45ab27ed9e701593af0a810db731143c"
server_key_iv: "2dcc0b588325a2bcaf159300fff410c1"
server_key_version: "encryptionkey@0"
I20260807 06:37:22.954773 3314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:22.979915 3314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:22.982129 3314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:22.982295 3314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:22.989025 3314 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41217
I20260807 06:37:22.989070 3419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41217 every 8 connection(s)
I20260807 06:37:22.989585 3314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:22.990737 3420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:22.993331 3420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57: Bootstrap starting.
I20260807 06:37:22.993989 3420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:22.994349 3420 log.cc:824] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:22.995100 3420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57: No bootstrap required, opened a new log
I20260807 06:37:22.996882 3420 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7412f3de04fd3b540341042553a57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260807 06:37:22.997032 3420 raft_consensus.cc:374] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:22.997058 3420 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec7412f3de04fd3b540341042553a57, State: Initialized, Role: FOLLOWER
I20260807 06:37:22.997242 3420 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7412f3de04fd3b540341042553a57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260807 06:37:22.997310 3420 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:22.997354 3420 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:22.997407 3420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:22.998018 3420 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7412f3de04fd3b540341042553a57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260807 06:37:22.998138 3420 leader_election.cc:302] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec7412f3de04fd3b540341042553a57; no voters:
I20260807 06:37:22.998335 3420 leader_election.cc:288] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:22.998404 3423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:22.998584 3423 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [term 1 LEADER]: Becoming Leader. State: Replica: cec7412f3de04fd3b540341042553a57, State: Running, Role: LEADER
I20260807 06:37:22.998638 3420 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:22.998706 3423 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7412f3de04fd3b540341042553a57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260807 06:37:22.999140 3427 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec7412f3de04fd3b540341042553a57. Latest consensus state: current_term: 1 leader_uuid: "cec7412f3de04fd3b540341042553a57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7412f3de04fd3b540341042553a57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } } }
I20260807 06:37:22.999295 3427 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:22.999284 3425 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cec7412f3de04fd3b540341042553a57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7412f3de04fd3b540341042553a57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } } }
I20260807 06:37:22.999671 3425 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:22.999751 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3314
I20260807 06:37:22.999856 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:22.999934 3442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:23.000061 3020 external_mini_cluster.cc:1428] Setting key 6f810dc7b45a3fb98520ab279d1b3e16
I20260807 06:37:23.000560 3442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 06:37:23 dist-test-slave-rwrg krb5kdc[3288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:37:23.007347 3442 catalog_manager.cc:1347] Generated new cluster ID: 29ff73e4b8fa48cf8a1c0026e807d6f9
I20260807 06:37:23.007411 3442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:23.011366 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:23.000598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58313 (local address 127.0.0.1:41217)
0807 06:37:23.001060 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:23.001069 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:23.001105 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:37:23.001287 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:23.001298 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:23.001362 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:23.001515 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:23.002524 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:23.003454 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:23.004208 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:23.004721 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:23.004842 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:23.007840 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:23.007844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:23.007845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:23.007856 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:23.009720 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:23.010143 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:23.010147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:23.010147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:23.010253 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:23.010498 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:23.010499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:23.010499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:23.010738 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:23.010930 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:23.011021 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 06:37:23.011140 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":90,"threads_started":1}
I20260807 06:37:23.012336 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:23.020043 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:23.013821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39113 (local address 127.0.0.1:41217)
0807 06:37:23.013860 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:23.013862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:23.013987 (+ 125us) server_negotiation.cc:407] Connection header received
0807 06:37:23.014050 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:23.014051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:23.014060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:23.014148 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:23.014674 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:23.015196 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:23.015908 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:23.016133 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:23.016210 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:23.017454 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:23.017455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:23.017456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:23.017463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:23.018880 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:23.019251 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:23.019253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:23.019253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:23.019312 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:23.019532 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:23.019533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:23.019533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:23.019643 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:23.019750 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:23.019871 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 06:37:23.019931 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:23.023101 3442 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:23.023895 3442 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:23.033882 3442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cec7412f3de04fd3b540341042553a57: Generated new TSK 0
I20260807 06:37:23.034122 3442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:23.041383 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3314
W20260807 06:37:23.057782 3286 connection.cc:535] client connection to 127.0.0.1:41217 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41217): Connection reset by peer (error 104)
2026-08-07T06:37:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1705 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:23 dist-test-slave-rwrg krb5kdc[3473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:23 dist-test-slave-rwrg krb5kdc[3473](info): set up 2 sockets
Aug 07 06:37:23 dist-test-slave-rwrg krb5kdc[3473](info): commencing operation
krb5kdc: starting...
W20260807 06:37:24.315058 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:24 dist-test-slave-rwrg krb5kdc[3473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:24.436772 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37989
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:24.518841 3493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:24.519102 3493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:24.519142 3493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:24.520283 3493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:24.520359 3493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:24.520381 3493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:24.520401 3493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:24.520431 3493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:24.521956 3493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37989
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:24.522246 3493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:24.522475 3493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:24.527452 3506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:24.527488 3493 server_base.cc:1034] running on GCE node
W20260807 06:37:24.527452 3500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:24.527535 3502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:24.527915 3493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:24.528165 3493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:24.529307 3493 hybrid_clock.cc:648] HybridClock initialized: now 1786084644529286 us; error 41 us; skew 500 ppm
Aug 07 06:37:24 dist-test-slave-rwrg krb5kdc[3473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:24.531908 3493 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:24.532359 3493 webserver.cc:460] Webserver started at http://127.0.0.1:46071/ using document root <none> and password file <none>
I20260807 06:37:24.532612 3493 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:24.532660 3493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:24.532771 3493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:24.533752 3493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "431d718c370e4dfe86f7c5f5f3fa1210"
format_stamp: "Formatted at 2026-08-07 06:37:24 on dist-test-slave-rwrg"
server_key: "a0e64e6282a1c3f1d2d6e9c88375b9b5"
server_key_iv: "0ce52afaef83a1fcf00550d10968b200"
server_key_version: "encryptionkey@0"
I20260807 06:37:24.534044 3493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "431d718c370e4dfe86f7c5f5f3fa1210"
format_stamp: "Formatted at 2026-08-07 06:37:24 on dist-test-slave-rwrg"
server_key: "a0e64e6282a1c3f1d2d6e9c88375b9b5"
server_key_iv: "0ce52afaef83a1fcf00550d10968b200"
server_key_version: "encryptionkey@0"
I20260807 06:37:24.535368 3493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:24.536110 3519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:24.536285 3493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:24.536396 3493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "431d718c370e4dfe86f7c5f5f3fa1210"
format_stamp: "Formatted at 2026-08-07 06:37:24 on dist-test-slave-rwrg"
server_key: "a0e64e6282a1c3f1d2d6e9c88375b9b5"
server_key_iv: "0ce52afaef83a1fcf00550d10968b200"
server_key_version: "encryptionkey@0"
I20260807 06:37:24.536471 3493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:24.557446 3493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:24.559948 3493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:24.560125 3493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:24.564332 3493 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37989
I20260807 06:37:24.564383 3609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37989 every 8 connection(s)
I20260807 06:37:24.564780 3493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:24.565776 3610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:24.568149 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210: Bootstrap starting.
I20260807 06:37:24.568779 3610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:24.569068 3610 log.cc:824] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:24.569893 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210: No bootstrap required, opened a new log
I20260807 06:37:24.571442 3610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260807 06:37:24.571583 3610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:24.571619 3610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431d718c370e4dfe86f7c5f5f3fa1210, State: Initialized, Role: FOLLOWER
I20260807 06:37:24.571760 3610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260807 06:37:24.571837 3610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:24.571859 3610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:24.571928 3610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:24.572103 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3493
I20260807 06:37:24.572222 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:24.572463 3020 external_mini_cluster.cc:1428] Setting key 8acc6448a88be9dbf8fcc3e2a95f939f
I20260807 06:37:24.572736 3610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260807 06:37:24.572829 3610 leader_election.cc:302] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 431d718c370e4dfe86f7c5f5f3fa1210; no voters:
I20260807 06:37:24.573012 3610 leader_election.cc:288] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:24.573064 3615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:24.573345 3615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [term 1 LEADER]: Becoming Leader. State: Replica: 431d718c370e4dfe86f7c5f5f3fa1210, State: Running, Role: LEADER
I20260807 06:37:24.573432 3615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260807 06:37:24.573891 3615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } } }
I20260807 06:37:24.573913 3619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 431d718c370e4dfe86f7c5f5f3fa1210. Latest consensus state: current_term: 1 leader_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431d718c370e4dfe86f7c5f5f3fa1210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } } }
I20260807 06:37:24.573983 3615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:24.573989 3619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:24.574322 3610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:24.574460 3622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:24.575023 3622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:24.576931 3622 catalog_manager.cc:1347] Generated new cluster ID: 1262331925e544619f1f7044bedb1177
I20260807 06:37:24.577034 3622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:24.578243 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:24.573017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46171 (local address 127.0.0.1:37989)
0807 06:37:24.573229 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:24.573238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:24.573276 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:37:24.573419 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:24.573425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:24.573466 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:24.573643 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:24.573822 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:24.575154 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:24.576244 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:24.576760 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:24.576894 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:24.577643 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:24.577647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:24.577647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:24.577823 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:24.577854 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:24.577864 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:24.577959 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:24.578043 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 06:37:24.578073 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260807 06:37:24.579255 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:24.591598 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:24.580629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41569 (local address 127.0.0.1:37989)
0807 06:37:24.580671 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:24.580673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:24.588731 (+ 8058us) server_negotiation.cc:407] Connection header received
0807 06:37:24.588801 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:24.588802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:24.588815 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:24.588861 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:24.589020 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:24.589537 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:24.590284 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:24.590504 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:24.590585 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:24.591306 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:24.591307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:24.591308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:24.591314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:24.591328 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:24.591332 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:24.591412 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:24.591479 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:37:24.591492 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:24.592129 3622 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:24.592823 3622 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:24.600476 3622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 431d718c370e4dfe86f7c5f5f3fa1210: Generated new TSK 0
I20260807 06:37:24.600718 3622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:24.608762 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3493
W20260807 06:37:24.628888 3471 connection.cc:535] client connection to 127.0.0.1:37989 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37989): Connection reset by peer (error 104)
2026-08-07T06:37:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:24 dist-test-slave-rwrg krb5kdc[3663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:24 dist-test-slave-rwrg krb5kdc[3663](info): set up 2 sockets
Aug 07 06:37:24 dist-test-slave-rwrg krb5kdc[3663](info): commencing operation
krb5kdc: starting...
W20260807 06:37:25.906714 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:25 dist-test-slave-rwrg krb5kdc[3663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:26.027522 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:26.109565 3680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:26.109807 3680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:26.109850 3680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:26.110975 3680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:26.111048 3680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:26.111073 3680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:26.111096 3680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:26.111124 3680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:26.112653 3680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41385
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:26.112936 3680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:26.113188 3680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:26.118377 3699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:26.118436 3691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:26.118453 3680 server_base.cc:1034] running on GCE node
W20260807 06:37:26.118400 3694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:26.118836 3680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:26.119103 3680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:26.120251 3680 hybrid_clock.cc:648] HybridClock initialized: now 1786084646120220 us; error 43 us; skew 500 ppm
Aug 07 06:37:26 dist-test-slave-rwrg krb5kdc[3663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:26.122874 3680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:26.123327 3680 webserver.cc:460] Webserver started at http://127.0.0.1:43027/ using document root <none> and password file <none>
I20260807 06:37:26.123531 3680 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:26.123575 3680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:26.123662 3680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:26.124733 3680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "2c477404f8374699ba51efb41d1ac655"
format_stamp: "Formatted at 2026-08-07 06:37:26 on dist-test-slave-rwrg"
server_key: "4e8d119434e194a1ee466aff65655912"
server_key_iv: "68dfd2e56d9ff1cc106fab085c6f675a"
server_key_version: "encryptionkey@0"
I20260807 06:37:26.125036 3680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "2c477404f8374699ba51efb41d1ac655"
format_stamp: "Formatted at 2026-08-07 06:37:26 on dist-test-slave-rwrg"
server_key: "4e8d119434e194a1ee466aff65655912"
server_key_iv: "68dfd2e56d9ff1cc106fab085c6f675a"
server_key_version: "encryptionkey@0"
I20260807 06:37:26.126307 3680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:26.127070 3708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:26.127251 3680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:26.127360 3680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "2c477404f8374699ba51efb41d1ac655"
format_stamp: "Formatted at 2026-08-07 06:37:26 on dist-test-slave-rwrg"
server_key: "4e8d119434e194a1ee466aff65655912"
server_key_iv: "68dfd2e56d9ff1cc106fab085c6f675a"
server_key_version: "encryptionkey@0"
I20260807 06:37:26.127434 3680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:26.130836 3680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:26.133164 3680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:26.133323 3680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:26.137427 3680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41385
I20260807 06:37:26.137461 3798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41385 every 8 connection(s)
I20260807 06:37:26.137850 3680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:26.138854 3799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:26.141218 3799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655: Bootstrap starting.
I20260807 06:37:26.141776 3799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:26.142055 3799 log.cc:824] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:26.142664 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3680
I20260807 06:37:26.142782 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:26.142976 3020 external_mini_cluster.cc:1428] Setting key 64a73bbe1ecbbe8bc46c40d54f4f7338
I20260807 06:37:26.142975 3799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655: No bootstrap required, opened a new log
I20260807 06:37:26.145290 3799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c477404f8374699ba51efb41d1ac655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260807 06:37:26.145457 3799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:26.145599 3799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c477404f8374699ba51efb41d1ac655, State: Initialized, Role: FOLLOWER
I20260807 06:37:26.145776 3799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c477404f8374699ba51efb41d1ac655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260807 06:37:26.145859 3799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:26.145871 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:26.143548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47533 (local address 127.0.0.1:41385)
0807 06:37:26.143759 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:26.143769 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:26.143804 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:37:26.144250 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:26.144257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:26.144282 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:26.144443 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:26.145192 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:26.145204 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:26.145204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:26.145208 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:26.145243 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:26.145253 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:26.145330 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:26.145401 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 06:37:26.145439 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260807 06:37:26.145881 3799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:26.146003 3799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:26.146709 3799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c477404f8374699ba51efb41d1ac655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260807 06:37:26.146837 3799 leader_election.cc:302] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c477404f8374699ba51efb41d1ac655; no voters:
I20260807 06:37:26.146848 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:26.147022 3799 leader_election.cc:288] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:26.147054 3808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:26.147169 3808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [term 1 LEADER]: Becoming Leader. State: Replica: 2c477404f8374699ba51efb41d1ac655, State: Running, Role: LEADER
I20260807 06:37:26.147289 3799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:26.147305 3808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c477404f8374699ba51efb41d1ac655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260807 06:37:26.147635 3808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c477404f8374699ba51efb41d1ac655. Latest consensus state: current_term: 1 leader_uuid: "2c477404f8374699ba51efb41d1ac655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c477404f8374699ba51efb41d1ac655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } } }
I20260807 06:37:26.147645 3809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c477404f8374699ba51efb41d1ac655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c477404f8374699ba51efb41d1ac655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } } }
I20260807 06:37:26.147766 3808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:26.147995 3809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655 [sys.catalog]: This master's current role is: LEADER
W20260807 06:37:26.149251 3842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:26.149320 3842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:26.149397 3826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:26.149585 3826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:26.150414 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:26.149151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45821 (local address 127.0.0.1:41385)
0807 06:37:26.149184 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:26.149186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:26.149398 (+ 212us) server_negotiation.cc:407] Connection header received
0807 06:37:26.149462 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:26.149463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:26.149468 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:26.149536 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:26.150162 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:26.150164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:26.150164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:26.150169 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:26.150184 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:26.150189 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:26.150249 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:26.150334 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 06:37:26.150350 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 06:37:26.151194 3826 catalog_manager.cc:1347] Generated new cluster ID: 44bb03c45032447594eb4e600d81b0a7
I20260807 06:37:26.151257 3826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:26.160367 3826 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:26.160580 3826 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:26.167716 3826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c477404f8374699ba51efb41d1ac655: Generated new TSK 0
I20260807 06:37:26.167927 3826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:26.170455 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3680
2026-08-07T06:37:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1555 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:26 dist-test-slave-rwrg krb5kdc[3851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:26 dist-test-slave-rwrg krb5kdc[3851](info): set up 2 sockets
Aug 07 06:37:26 dist-test-slave-rwrg krb5kdc[3851](info): commencing operation
krb5kdc: starting...
W20260807 06:37:27.430665 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:27 dist-test-slave-rwrg krb5kdc[3851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:27.553179 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:27.634310 3879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:27.634550 3879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:27.634601 3879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:27.635751 3879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:27.635823 3879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:27.635849 3879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:27.635882 3879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:27.635908 3879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:27.637499 3879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:27.637796 3879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:27.638070 3879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:27.643172 3890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:27.643323 3879 server_base.cc:1034] running on GCE node
W20260807 06:37:27.643172 3886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:27.643462 3887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:27.643728 3879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:27.644089 3879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:27.645254 3879 hybrid_clock.cc:648] HybridClock initialized: now 1786084647645238 us; error 31 us; skew 500 ppm
Aug 07 06:37:27 dist-test-slave-rwrg krb5kdc[3851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:27.652397 3879 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:27.652889 3879 webserver.cc:460] Webserver started at http://127.0.0.1:33349/ using document root <none> and password file <none>
I20260807 06:37:27.653169 3879 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:27.653244 3879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:27.653393 3879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:27.654477 3879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "2ed0712da42b49de90abbf416295a510"
format_stamp: "Formatted at 2026-08-07 06:37:27 on dist-test-slave-rwrg"
server_key: "6f1c0c06883166e2f81e32c527936acd"
server_key_iv: "fd90e9f73f1a46743f631db288bf2a31"
server_key_version: "encryptionkey@0"
I20260807 06:37:27.654838 3879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "2ed0712da42b49de90abbf416295a510"
format_stamp: "Formatted at 2026-08-07 06:37:27 on dist-test-slave-rwrg"
server_key: "6f1c0c06883166e2f81e32c527936acd"
server_key_iv: "fd90e9f73f1a46743f631db288bf2a31"
server_key_version: "encryptionkey@0"
I20260807 06:37:27.656189 3879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:27.657186 3900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:27.657400 3879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:27.657510 3879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "2ed0712da42b49de90abbf416295a510"
format_stamp: "Formatted at 2026-08-07 06:37:27 on dist-test-slave-rwrg"
server_key: "6f1c0c06883166e2f81e32c527936acd"
server_key_iv: "fd90e9f73f1a46743f631db288bf2a31"
server_key_version: "encryptionkey@0"
I20260807 06:37:27.657598 3879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:27.679936 3879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:27.682291 3879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:27.682413 3879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:27.686478 3879 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44269
I20260807 06:37:27.686527 3982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44269 every 8 connection(s)
I20260807 06:37:27.686901 3879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:27.688081 3985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:27.688604 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 3879
I20260807 06:37:27.688721 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:27.688915 3020 external_mini_cluster.cc:1428] Setting key 4536262ca21b4cc8d23418ef0db940e7
I20260807 06:37:27.691372 3985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510: Bootstrap starting.
I20260807 06:37:27.691988 3985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:27.692379 3985 log.cc:824] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:27.693186 3985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510: No bootstrap required, opened a new log
Aug 07 06:37:27 dist-test-slave-rwrg krb5kdc[3851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:37:27.695681 3985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed0712da42b49de90abbf416295a510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260807 06:37:27.695881 3985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:27.695931 3985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed0712da42b49de90abbf416295a510, State: Initialized, Role: FOLLOWER
I20260807 06:37:27.696065 3985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed0712da42b49de90abbf416295a510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260807 06:37:27.696157 3985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:27.696214 3985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:27.696266 3985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:27.696893 3985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed0712da42b49de90abbf416295a510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260807 06:37:27.697016 3985 leader_election.cc:302] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ed0712da42b49de90abbf416295a510; no voters:
I20260807 06:37:27.697188 3985 leader_election.cc:288] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:27.697346 3990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:27.697470 3985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:27.697589 3990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [term 1 LEADER]: Becoming Leader. State: Replica: 2ed0712da42b49de90abbf416295a510, State: Running, Role: LEADER
I20260807 06:37:27.697743 3990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed0712da42b49de90abbf416295a510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260807 06:37:27.698105 3991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ed0712da42b49de90abbf416295a510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed0712da42b49de90abbf416295a510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } } }
I20260807 06:37:27.698195 3991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:27.698369 3994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ed0712da42b49de90abbf416295a510. Latest consensus state: current_term: 1 leader_uuid: "2ed0712da42b49de90abbf416295a510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed0712da42b49de90abbf416295a510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } } }
I20260807 06:37:27.698436 3994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:27.698864 4006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:27.699541 4006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:27.700057 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:27.689487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42093 (local address 127.0.0.1:44269)
0807 06:37:27.689701 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:27.689711 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:27.689746 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:37:27.690154 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:27.690161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:27.690206 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:27.690321 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:27.691154 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:27.692123 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:27.693117 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:27.693535 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:27.693615 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:27.696244 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:27.696248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:27.696248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:27.696260 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:27.698304 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:27.698753 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:27.698756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:27.698756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:27.698867 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:27.699081 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:27.699083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:27.699083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:27.699381 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:27.699562 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:27.699711 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 06:37:27.699853 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":115,"threads_started":1}
I20260807 06:37:27.701097 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:27.701192 4006 catalog_manager.cc:1347] Generated new cluster ID: d4e91f44374f4c618e554b3a15995f1f
I20260807 06:37:27.701236 4006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:27.705801 4006 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:27.706499 4006 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:27.721674 4006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ed0712da42b49de90abbf416295a510: Generated new TSK 0
I20260807 06:37:27.721917 4006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:27.725378 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:27.705192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45915 (local address 127.0.0.1:44269)
0807 06:37:27.708437 (+ 3245us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:27.708440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:27.708450 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:27.708517 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:27.708518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:27.708529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:27.708621 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:27.709792 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:27.710531 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:27.711617 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:27.711946 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:27.711996 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:27.721480 (+ 9484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:27.721481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:27.721481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:27.721489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:27.724097 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:27.724542 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:27.724542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:27.724543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:27.724609 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:27.724900 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:27.724902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:27.724902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:27.725022 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:27.725111 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:27.725257 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 06:37:27.725320 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3219}
I20260807 06:37:27.727566 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 3879
2026-08-07T06:37:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1557 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:27 dist-test-slave-rwrg krb5kdc[4029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:27 dist-test-slave-rwrg krb5kdc[4029](info): set up 2 sockets
Aug 07 06:37:27 dist-test-slave-rwrg krb5kdc[4029](info): commencing operation
krb5kdc: starting...
W20260807 06:37:29.102025 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:29 dist-test-slave-rwrg krb5kdc[4029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:29.222581 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35213
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:29.302908 4047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:29.303131 4047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:29.303164 4047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:29.304250 4047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:29.304292 4047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:29.304306 4047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:29.304318 4047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:29.304330 4047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:29.305799 4047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:29.306038 4047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:29.306248 4047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:29.311192 4062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:29.311216 4065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:29.311216 4063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:29.311600 4047 server_base.cc:1034] running on GCE node
I20260807 06:37:29.311828 4047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:29.312150 4047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:29.313293 4047 hybrid_clock.cc:648] HybridClock initialized: now 1786084649313285 us; error 32 us; skew 500 ppm
Aug 07 06:37:29 dist-test-slave-rwrg krb5kdc[4029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:29.315901 4047 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:29.316401 4047 webserver.cc:460] Webserver started at http://127.0.0.1:45663/ using document root <none> and password file <none>
I20260807 06:37:29.316656 4047 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:29.316705 4047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:29.316860 4047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:29.317912 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "71c459431d7243f0a5b27d8475b9794e"
format_stamp: "Formatted at 2026-08-07 06:37:29 on dist-test-slave-rwrg"
server_key: "2c33c6ee0bbcdf3ed0de0c9b19ce4548"
server_key_iv: "16519971ce88fcc991e2d9a557e094ce"
server_key_version: "encryptionkey@0"
I20260807 06:37:29.318295 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "71c459431d7243f0a5b27d8475b9794e"
format_stamp: "Formatted at 2026-08-07 06:37:29 on dist-test-slave-rwrg"
server_key: "2c33c6ee0bbcdf3ed0de0c9b19ce4548"
server_key_iv: "16519971ce88fcc991e2d9a557e094ce"
server_key_version: "encryptionkey@0"
I20260807 06:37:29.319669 4047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:29.320423 4075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:29.320645 4047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:29.320755 4047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "71c459431d7243f0a5b27d8475b9794e"
format_stamp: "Formatted at 2026-08-07 06:37:29 on dist-test-slave-rwrg"
server_key: "2c33c6ee0bbcdf3ed0de0c9b19ce4548"
server_key_iv: "16519971ce88fcc991e2d9a557e094ce"
server_key_version: "encryptionkey@0"
I20260807 06:37:29.320829 4047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:29.342208 4047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:29.344403 4047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:29.344609 4047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:29.348798 4047 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35213
I20260807 06:37:29.348852 4160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35213 every 8 connection(s)
I20260807 06:37:29.349255 4047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:29.350342 4162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:29.352707 4162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e: Bootstrap starting.
I20260807 06:37:29.353292 4162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:29.353580 4162 log.cc:824] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:29.354367 4162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e: No bootstrap required, opened a new log
I20260807 06:37:29.356056 4162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c459431d7243f0a5b27d8475b9794e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35213 } }
I20260807 06:37:29.356194 4162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:29.356216 4162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71c459431d7243f0a5b27d8475b9794e, State: Initialized, Role: FOLLOWER
I20260807 06:37:29.356397 4162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c459431d7243f0a5b27d8475b9794e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35213 } }
I20260807 06:37:29.356462 4162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:29.356537 4162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:29.356592 4162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:29.357151 4162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c459431d7243f0a5b27d8475b9794e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35213 } }
I20260807 06:37:29.357239 4162 leader_election.cc:302] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71c459431d7243f0a5b27d8475b9794e; no voters:
I20260807 06:37:29.357474 4162 leader_election.cc:288] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:29.357504 4167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:29.357590 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4047
I20260807 06:37:29.357623 4167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [term 1 LEADER]: Becoming Leader. State: Replica: 71c459431d7243f0a5b27d8475b9794e, State: Running, Role: LEADER
I20260807 06:37:29.357719 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:29.357707 4167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c459431d7243f0a5b27d8475b9794e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35213 } }
I20260807 06:37:29.357736 4162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:29.357895 3020 external_mini_cluster.cc:1428] Setting key 0619ecc42196f514faf426b133e46f62
I20260807 06:37:29.358012 4167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71c459431d7243f0a5b27d8475b9794e. Latest consensus state: current_term: 1 leader_uuid: "71c459431d7243f0a5b27d8475b9794e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c459431d7243f0a5b27d8475b9794e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35213 } } }
I20260807 06:37:29.358141 4167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:29.358235 4168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71c459431d7243f0a5b27d8475b9794e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71c459431d7243f0a5b27d8475b9794e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35213 } } }
I20260807 06:37:29.358347 4168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:29.358439 4176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:29.359645 4176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:29.361186 4176 catalog_manager.cc:1347] Generated new cluster ID: ebff3c00c27e45bca0b66ca6d36f79c5
I20260807 06:37:29.361240 4176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:29.364117 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:29.358823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57333 (local address 127.0.0.1:35213)
0807 06:37:29.359148 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:29.359164 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:29.359217 (+ 53us) server_negotiation.cc:407] Connection header received
0807 06:37:29.359566 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:29.359572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:29.359618 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:29.359781 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:29.360122 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:29.361454 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:29.362264 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:29.362676 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:29.362800 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:29.363499 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:29.363503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:29.363503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:29.363683 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:29.363712 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:29.363723 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:29.363800 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:29.363879 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 06:37:29.363945 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":252,"threads_started":1}
I20260807 06:37:29.365093 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:29.369699 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:29.366588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41957 (local address 127.0.0.1:35213)
0807 06:37:29.366641 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:29.366643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:29.366812 (+ 169us) server_negotiation.cc:407] Connection header received
0807 06:37:29.366836 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:29.366836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:29.366847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:29.366902 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:29.367041 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:29.367482 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:29.368442 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:29.368790 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:29.368850 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:29.369449 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:29.369450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:29.369450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:29.369455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:29.369467 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:29.369471 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:29.369541 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:29.369583 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 06:37:29.369602 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:37:29.372238 4176 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:29.372884 4176 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:29.376312 4176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71c459431d7243f0a5b27d8475b9794e: Generated new TSK 0
I20260807 06:37:29.376498 4176 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:29.378562 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4047
2026-08-07T06:37:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:29 dist-test-slave-rwrg krb5kdc[4205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:29 dist-test-slave-rwrg krb5kdc[4205](info): set up 2 sockets
Aug 07 06:37:29 dist-test-slave-rwrg krb5kdc[4205](info): commencing operation
krb5kdc: starting...
W20260807 06:37:30.636157 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:30 dist-test-slave-rwrg krb5kdc[4205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:30.756780 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:30.837884 4227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:30.838138 4227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:30.838183 4227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:30.839319 4227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:30.839388 4227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:30.839412 4227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:30.839434 4227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:30.839463 4227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:30.840950 4227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:30.841236 4227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:30.841467 4227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:30.846400 4234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:30.846449 4239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:30.846400 4233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:30.846621 4227 server_base.cc:1034] running on GCE node
I20260807 06:37:30.846871 4227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:30.847162 4227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:30.848333 4227 hybrid_clock.cc:648] HybridClock initialized: now 1786084650848322 us; error 46 us; skew 500 ppm
Aug 07 06:37:30 dist-test-slave-rwrg krb5kdc[4205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084650, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:30.850929 4227 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:30.851426 4227 webserver.cc:460] Webserver started at http://127.0.0.1:44127/ using document root <none> and password file <none>
I20260807 06:37:30.851656 4227 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:30.851729 4227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:30.851883 4227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:30.852994 4227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "4aa0413fafe84089ba326e8b5a75f561"
format_stamp: "Formatted at 2026-08-07 06:37:30 on dist-test-slave-rwrg"
server_key: "6c7236e9965b5734fa8eaee7deaa869d"
server_key_iv: "940e56c890f419aeebaae70740cce104"
server_key_version: "encryptionkey@0"
I20260807 06:37:30.853359 4227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "4aa0413fafe84089ba326e8b5a75f561"
format_stamp: "Formatted at 2026-08-07 06:37:30 on dist-test-slave-rwrg"
server_key: "6c7236e9965b5734fa8eaee7deaa869d"
server_key_iv: "940e56c890f419aeebaae70740cce104"
server_key_version: "encryptionkey@0"
I20260807 06:37:30.854712 4227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:30.855412 4248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:30.855585 4227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:30.855731 4227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "4aa0413fafe84089ba326e8b5a75f561"
format_stamp: "Formatted at 2026-08-07 06:37:30 on dist-test-slave-rwrg"
server_key: "6c7236e9965b5734fa8eaee7deaa869d"
server_key_iv: "940e56c890f419aeebaae70740cce104"
server_key_version: "encryptionkey@0"
I20260807 06:37:30.855809 4227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:30.859141 4227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:30.861500 4227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:30.861660 4227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:30.865854 4227 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38695
I20260807 06:37:30.865887 4324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38695 every 8 connection(s)
I20260807 06:37:30.866241 4227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:30.867252 4327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:30.869691 4327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561: Bootstrap starting.
I20260807 06:37:30.870282 4327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:30.870568 4327 log.cc:824] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:30.871281 4327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561: No bootstrap required, opened a new log
I20260807 06:37:30.871577 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4227
I20260807 06:37:30.871692 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:30.871881 3020 external_mini_cluster.cc:1428] Setting key 46581cc3bc717d1ed0a484cdf480acb7
I20260807 06:37:30.873245 4327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0413fafe84089ba326e8b5a75f561" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 06:37:30.873401 4327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:30.873466 4327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa0413fafe84089ba326e8b5a75f561, State: Initialized, Role: FOLLOWER
I20260807 06:37:30.873641 4327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0413fafe84089ba326e8b5a75f561" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 06:37:30.873739 4327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:30.873785 4327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:30.873842 4327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:30.874717 4327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0413fafe84089ba326e8b5a75f561" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 06:37:30.874747 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:30.872401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49427 (local address 127.0.0.1:38695)
0807 06:37:30.872619 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:30.872628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:30.872661 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:37:30.873259 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:30.873268 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:30.873299 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:30.873479 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:30.874273 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:30.874276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:30.874277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:30.874287 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:30.874323 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:30.874333 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:30.874403 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:30.874478 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 06:37:30.874615 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":108,"threads_started":1}
I20260807 06:37:30.874804 4327 leader_election.cc:302] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aa0413fafe84089ba326e8b5a75f561; no voters:
I20260807 06:37:30.874990 4327 leader_election.cc:288] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:30.875103 4336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:30.875252 4336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa0413fafe84089ba326e8b5a75f561, State: Running, Role: LEADER
I20260807 06:37:30.875303 4327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:30.875430 4336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0413fafe84089ba326e8b5a75f561" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260807 06:37:30.875815 4336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aa0413fafe84089ba326e8b5a75f561. Latest consensus state: current_term: 1 leader_uuid: "4aa0413fafe84089ba326e8b5a75f561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0413fafe84089ba326e8b5a75f561" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260807 06:37:30.875912 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:30.875825 4342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aa0413fafe84089ba326e8b5a75f561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0413fafe84089ba326e8b5a75f561" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260807 06:37:30.875970 4342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:30.876111 4336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:30.876210 4353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:30.877190 4353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:30.878968 4353 catalog_manager.cc:1347] Generated new cluster ID: d00b8e5c9b524e52ab42c2702ea4048e
I20260807 06:37:30.879046 4353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:30.879101 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:30.877621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47177 (local address 127.0.0.1:38695)
0807 06:37:30.877672 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:30.877675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:30.877937 (+ 262us) server_negotiation.cc:407] Connection header received
0807 06:37:30.878046 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:30.878046 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:30.878051 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:30.878079 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:30.878791 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:30.878792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:30.878793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:30.878797 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:30.878809 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:30.878814 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:30.878845 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:30.878950 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 06:37:30.878963 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 06:37:30.887729 4353 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:30.887873 4353 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:30.904105 4353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aa0413fafe84089ba326e8b5a75f561: Generated new TSK 0
I20260807 06:37:30.904328 4353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:30.912453 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4227
2026-08-07T06:37:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:30 dist-test-slave-rwrg krb5kdc[4374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:30 dist-test-slave-rwrg krb5kdc[4374](info): set up 2 sockets
Aug 07 06:37:30 dist-test-slave-rwrg krb5kdc[4374](info): commencing operation
krb5kdc: starting...
W20260807 06:37:32.231472 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:32 dist-test-slave-rwrg krb5kdc[4374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:32.369563 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:32.455540 4392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:32.455747 4392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:32.455772 4392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:32.457132 4392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:32.457183 4392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:32.457198 4392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:32.457211 4392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:32.457259 4392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:32.458906 4392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:32.459146 4392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:32.459372 4392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:32.464596 4400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:32.464560 4403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:32.464643 4399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:32.464651 4392 server_base.cc:1034] running on GCE node
I20260807 06:37:32.465016 4392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:32.465296 4392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:32.466461 4392 hybrid_clock.cc:648] HybridClock initialized: now 1786084652466435 us; error 59 us; skew 500 ppm
Aug 07 06:37:32 dist-test-slave-rwrg krb5kdc[4374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084652, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:32.469071 4392 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:32.469558 4392 webserver.cc:460] Webserver started at http://127.0.0.1:35417/ using document root <none> and password file <none>
I20260807 06:37:32.469801 4392 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:32.469868 4392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:32.470036 4392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:32.471064 4392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "c25f762a4571442986f00e1dd59c291b"
format_stamp: "Formatted at 2026-08-07 06:37:32 on dist-test-slave-rwrg"
server_key: "f7cbd00bfad82bb98745f9aa00a85155"
server_key_iv: "8eca21ac08f914405d6a5b3f40506e41"
server_key_version: "encryptionkey@0"
I20260807 06:37:32.471421 4392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "c25f762a4571442986f00e1dd59c291b"
format_stamp: "Formatted at 2026-08-07 06:37:32 on dist-test-slave-rwrg"
server_key: "f7cbd00bfad82bb98745f9aa00a85155"
server_key_iv: "8eca21ac08f914405d6a5b3f40506e41"
server_key_version: "encryptionkey@0"
I20260807 06:37:32.472833 4392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:37:32.473585 4417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:32.473759 4392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 06:37:32.473882 4392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "c25f762a4571442986f00e1dd59c291b"
format_stamp: "Formatted at 2026-08-07 06:37:32 on dist-test-slave-rwrg"
server_key: "f7cbd00bfad82bb98745f9aa00a85155"
server_key_iv: "8eca21ac08f914405d6a5b3f40506e41"
server_key_version: "encryptionkey@0"
I20260807 06:37:32.473953 4392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:32.494500 4392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:32.496847 4392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:32.497025 4392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:32.501036 4392 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42137
I20260807 06:37:32.501073 4499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42137 every 8 connection(s)
I20260807 06:37:32.501430 4392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:32.502406 4500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:32.504751 4500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b: Bootstrap starting.
I20260807 06:37:32.505030 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4392
I20260807 06:37:32.505164 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:32.505354 3020 external_mini_cluster.cc:1428] Setting key dde1fa21d0f20193ad6fd3802a827b7f
I20260807 06:37:32.505555 4500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:32.505877 4500 log.cc:824] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:32.506699 4500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b: No bootstrap required, opened a new log
I20260807 06:37:32.508597 4500 raft_consensus.cc:348] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25f762a4571442986f00e1dd59c291b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42137 } }
I20260807 06:37:32.508759 4500 raft_consensus.cc:374] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:32.508807 4500 raft_consensus.cc:729] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25f762a4571442986f00e1dd59c291b, State: Initialized, Role: FOLLOWER
I20260807 06:37:32.508934 4500 consensus_queue.cc:260] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25f762a4571442986f00e1dd59c291b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42137 } }
I20260807 06:37:32.509032 4500 raft_consensus.cc:388] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:32.509076 4500 raft_consensus.cc:482] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:32.509124 4500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:32.509706 4500 raft_consensus.cc:504] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25f762a4571442986f00e1dd59c291b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42137 } }
I20260807 06:37:32.509830 4500 leader_election.cc:302] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c25f762a4571442986f00e1dd59c291b; no voters:
I20260807 06:37:32.510032 4500 leader_election.cc:288] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:32.510080 4510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:32.510239 4510 raft_consensus.cc:686] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [term 1 LEADER]: Becoming Leader. State: Replica: c25f762a4571442986f00e1dd59c291b, State: Running, Role: LEADER
I20260807 06:37:32.510305 4500 sys_catalog.cc:564] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:32.510421 4510 consensus_queue.cc:237] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25f762a4571442986f00e1dd59c291b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42137 } }
I20260807 06:37:32.510789 4514 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c25f762a4571442986f00e1dd59c291b. Latest consensus state: current_term: 1 leader_uuid: "c25f762a4571442986f00e1dd59c291b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25f762a4571442986f00e1dd59c291b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42137 } } }
I20260807 06:37:32.510792 4513 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c25f762a4571442986f00e1dd59c291b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25f762a4571442986f00e1dd59c291b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42137 } } }
I20260807 06:37:32.510982 4514 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:32.511195 4513 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:32.511617 4531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:32.512212 4531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 06:37:32 dist-test-slave-rwrg krb5kdc[4374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:37:32.514724 4531 catalog_manager.cc:1347] Generated new cluster ID: d318c67ad1a142c48d63b91f77263a11
I20260807 06:37:32.514788 4531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:32.517562 4509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:32.506055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:43995 (local address 127.0.0.1:42137)
0807 06:37:32.506277 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:32.506286 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:32.506321 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:37:32.506814 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:32.506823 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:32.506893 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:32.507044 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:32.507674 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:32.508624 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:32.509566 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:32.510136 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:32.510580 (+ 444us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:32.513815 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:32.513819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:32.513819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:32.513831 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:32.515810 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:32.516305 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:32.516308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:32.516308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:32.516411 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:32.516692 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:32.516693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:32.516693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:32.516973 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:32.517134 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:32.517277 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 06:37:32.517373 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":116,"threads_started":1}
I20260807 06:37:32.518501 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:32.521437 4531 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:32.522095 4531 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:32.526995 4509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:32.519952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:46913 (local address 127.0.0.1:42137)
0807 06:37:32.519992 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:32.519994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:32.520104 (+ 110us) server_negotiation.cc:407] Connection header received
0807 06:37:32.520136 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:32.520137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:32.520147 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:32.520214 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:32.520839 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:32.521591 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:32.522566 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:32.522871 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:32.522944 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:32.524248 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:32.524250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:32.524250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:32.524257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:32.525811 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:32.526226 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:32.526227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:32.526227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:32.526289 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:32.526502 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:32.526503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:32.526504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:32.526624 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:32.526723 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:32.526835 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 06:37:32.526898 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:37:32.536541 4531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c25f762a4571442986f00e1dd59c291b: Generated new TSK 0
I20260807 06:37:32.536726 4531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:32.545871 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4392
W20260807 06:37:32.557428 4372 connection.cc:535] client connection to 127.0.0.1:42137 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42137): Connection reset by peer (error 104)
2026-08-07T06:37:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1633 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:32 dist-test-slave-rwrg krb5kdc[4557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:32 dist-test-slave-rwrg krb5kdc[4557](info): set up 2 sockets
Aug 07 06:37:32 dist-test-slave-rwrg krb5kdc[4557](info): commencing operation
krb5kdc: starting...
W20260807 06:37:33.820961 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:33 dist-test-slave-rwrg krb5kdc[4557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:33.947646 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:34.030261 4582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:34.030467 4582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:34.030498 4582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:34.031656 4582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:34.031700 4582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:34.031714 4582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:34.031728 4582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:34.031740 4582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:34.033339 4582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:34.033619 4582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:34.033855 4582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:34.039076 4594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:34.039141 4589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:34.039076 4588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:34.039279 4582 server_base.cc:1034] running on GCE node
I20260807 06:37:34.039593 4582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:34.039846 4582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:34.041033 4582 hybrid_clock.cc:648] HybridClock initialized: now 1786084654041005 us; error 47 us; skew 500 ppm
Aug 07 06:37:34 dist-test-slave-rwrg krb5kdc[4557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084654, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:34.047662 4582 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:34.048149 4582 webserver.cc:460] Webserver started at http://127.0.0.1:45481/ using document root <none> and password file <none>
I20260807 06:37:34.048342 4582 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:34.048388 4582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:34.048480 4582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:34.049579 4582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "565c4162399c4682b95ef07d60d31eac"
format_stamp: "Formatted at 2026-08-07 06:37:34 on dist-test-slave-rwrg"
server_key: "7781271742bc0bc9e33a9212ad28ff90"
server_key_iv: "545a332ea7fc64b9f65c04feb6efad3b"
server_key_version: "encryptionkey@0"
I20260807 06:37:34.049885 4582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "565c4162399c4682b95ef07d60d31eac"
format_stamp: "Formatted at 2026-08-07 06:37:34 on dist-test-slave-rwrg"
server_key: "7781271742bc0bc9e33a9212ad28ff90"
server_key_iv: "545a332ea7fc64b9f65c04feb6efad3b"
server_key_version: "encryptionkey@0"
I20260807 06:37:34.051179 4582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 06:37:34.052011 4602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:34.052245 4582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:34.052340 4582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "565c4162399c4682b95ef07d60d31eac"
format_stamp: "Formatted at 2026-08-07 06:37:34 on dist-test-slave-rwrg"
server_key: "7781271742bc0bc9e33a9212ad28ff90"
server_key_iv: "545a332ea7fc64b9f65c04feb6efad3b"
server_key_version: "encryptionkey@0"
I20260807 06:37:34.052395 4582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:34.071372 4582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:34.073894 4582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:34.074020 4582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:34.077989 4582 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32897
I20260807 06:37:34.078064 4692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32897 every 8 connection(s)
I20260807 06:37:34.078418 4582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:34.079537 4694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:34.082036 4694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac: Bootstrap starting.
I20260807 06:37:34.082671 4694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:34.082981 4694 log.cc:824] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:34.083421 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4582
I20260807 06:37:34.083545 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:34.083783 3020 external_mini_cluster.cc:1428] Setting key 5dab0d3d689621e3c910b8388702d5ba
I20260807 06:37:34.083909 4694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac: No bootstrap required, opened a new log
I20260807 06:37:34.085736 4694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c4162399c4682b95ef07d60d31eac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260807 06:37:34.085897 4694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:34.085963 4694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565c4162399c4682b95ef07d60d31eac, State: Initialized, Role: FOLLOWER
I20260807 06:37:34.086134 4694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c4162399c4682b95ef07d60d31eac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260807 06:37:34.086236 4694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:34.086282 4694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:34.086349 4694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:34.086894 4694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c4162399c4682b95ef07d60d31eac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260807 06:37:34.087013 4694 leader_election.cc:302] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 565c4162399c4682b95ef07d60d31eac; no voters:
I20260807 06:37:34.087229 4694 leader_election.cc:288] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:34.087273 4699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:34.087476 4694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:34.087558 4699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [term 1 LEADER]: Becoming Leader. State: Replica: 565c4162399c4682b95ef07d60d31eac, State: Running, Role: LEADER
I20260807 06:37:34.087700 4699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c4162399c4682b95ef07d60d31eac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260807 06:37:34.088076 4699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565c4162399c4682b95ef07d60d31eac. Latest consensus state: current_term: 1 leader_uuid: "565c4162399c4682b95ef07d60d31eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c4162399c4682b95ef07d60d31eac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260807 06:37:34.088191 4699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:34.088929 4700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "565c4162399c4682b95ef07d60d31eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c4162399c4682b95ef07d60d31eac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260807 06:37:34.089013 4700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:34.089218 4711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:34.089901 4711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:34.091540 4711 catalog_manager.cc:1347] Generated new cluster ID: d326b1d9d2d34bf1a23b88bda72d1443
I20260807 06:37:34.091578 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:34.084335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:52737 (local address 127.0.0.1:32897)
0807 06:37:34.084550 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:34.084558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:34.084598 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:37:34.085736 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:34.085745 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:34.085808 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:34.086015 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:34.086236 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:34.087811 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:34.089171 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:34.089854 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:34.089979 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:34.090764 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:34.090768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:34.090768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:34.090992 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:34.091031 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:34.091048 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:34.091143 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:34.091298 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 06:37:34.091328 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260807 06:37:34.091598 4711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:34.092577 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:34.097708 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:34.094027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:56713 (local address 127.0.0.1:32897)
0807 06:37:34.094061 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:34.094063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:34.094214 (+ 151us) server_negotiation.cc:407] Connection header received
0807 06:37:34.094246 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:34.094247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:34.094280 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:34.094344 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:34.094505 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:34.095214 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:34.096193 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:34.096621 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:34.096717 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:34.097428 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:34.097429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:34.097429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:34.097433 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:34.097447 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:34.097451 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:34.097495 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:34.097613 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 06:37:34.097631 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:37:34.106902 4711 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:34.107427 4711 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:34.110888 4711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 565c4162399c4682b95ef07d60d31eac: Generated new TSK 0
I20260807 06:37:34.111088 4711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:34.116492 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4582
W20260807 06:37:34.132242 4555 connection.cc:535] client connection to 127.0.0.1:32897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32897): Connection reset by peer (error 104)
2026-08-07T06:37:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:34 dist-test-slave-rwrg krb5kdc[4745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:34 dist-test-slave-rwrg krb5kdc[4745](info): set up 2 sockets
Aug 07 06:37:34 dist-test-slave-rwrg krb5kdc[4745](info): commencing operation
krb5kdc: starting...
W20260807 06:37:35.415432 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:35 dist-test-slave-rwrg krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:35.535539 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:35.619112 4764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:35.619310 4764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:35.619333 4764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:35.620545 4764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:35.620584 4764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:35.620599 4764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:35.620610 4764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:35.620622 4764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:35.622051 4764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:35.622295 4764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:35.622519 4764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:35.627348 4773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:35.627344 4769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:35.627586 4764 server_base.cc:1034] running on GCE node
W20260807 06:37:35.627386 4770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:35.627887 4764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:35.628118 4764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:35.629266 4764 hybrid_clock.cc:648] HybridClock initialized: now 1786084655629259 us; error 54 us; skew 500 ppm
Aug 07 06:37:35 dist-test-slave-rwrg krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:35.631845 4764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:35.632294 4764 webserver.cc:460] Webserver started at http://127.0.0.1:38819/ using document root <none> and password file <none>
I20260807 06:37:35.632622 4764 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:35.632691 4764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:35.632824 4764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:35.633912 4764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "cdc814f2198148daa1d3c02da8ea09d2"
format_stamp: "Formatted at 2026-08-07 06:37:35 on dist-test-slave-rwrg"
server_key: "8a5d9e78f092fe04e44e8e2c12cd2b09"
server_key_iv: "3d0a43f88a229da6b254467cc508a4df"
server_key_version: "encryptionkey@0"
I20260807 06:37:35.634315 4764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "cdc814f2198148daa1d3c02da8ea09d2"
format_stamp: "Formatted at 2026-08-07 06:37:35 on dist-test-slave-rwrg"
server_key: "8a5d9e78f092fe04e44e8e2c12cd2b09"
server_key_iv: "3d0a43f88a229da6b254467cc508a4df"
server_key_version: "encryptionkey@0"
I20260807 06:37:35.635661 4764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:35.636651 4785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:35.636880 4764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:35.637003 4764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "cdc814f2198148daa1d3c02da8ea09d2"
format_stamp: "Formatted at 2026-08-07 06:37:35 on dist-test-slave-rwrg"
server_key: "8a5d9e78f092fe04e44e8e2c12cd2b09"
server_key_iv: "3d0a43f88a229da6b254467cc508a4df"
server_key_version: "encryptionkey@0"
I20260807 06:37:35.637090 4764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:35.640558 4764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:35.642597 4764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:35.642763 4764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:35.647058 4764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45009
I20260807 06:37:35.647094 4862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45009 every 8 connection(s)
I20260807 06:37:35.647452 4764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:35.648411 4863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:35.650677 4863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2: Bootstrap starting.
I20260807 06:37:35.650872 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4764
I20260807 06:37:35.651014 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:35.651209 3020 external_mini_cluster.cc:1428] Setting key a077b452dab8d42ece64a40638e70123
I20260807 06:37:35.651376 4863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:35.651743 4863 log.cc:824] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:35.652554 4863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2: No bootstrap required, opened a new log
I20260807 06:37:35.653821 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:35.651705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:55927 (local address 127.0.0.1:45009)
0807 06:37:35.651971 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:35.651980 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:35.652012 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:37:35.652569 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:35.652579 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:35.652611 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:35.652760 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:35.653409 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:35.653413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:35.653413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:35.653417 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:35.653457 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:35.653471 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:35.653564 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:35.653593 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 06:37:35.653632 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":100,"threads_started":1}
I20260807 06:37:35.654973 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:35.654964 4863 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc814f2198148daa1d3c02da8ea09d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260807 06:37:35.655123 4863 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:35.655162 4863 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc814f2198148daa1d3c02da8ea09d2, State: Initialized, Role: FOLLOWER
I20260807 06:37:35.655310 4863 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc814f2198148daa1d3c02da8ea09d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260807 06:37:35.655392 4863 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:35.655421 4863 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:35.655473 4863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:35.656191 4863 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc814f2198148daa1d3c02da8ea09d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260807 06:37:35.656312 4863 leader_election.cc:302] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdc814f2198148daa1d3c02da8ea09d2; no voters:
I20260807 06:37:35.656478 4863 leader_election.cc:288] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:35.656554 4882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:35.656689 4882 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [term 1 LEADER]: Becoming Leader. State: Replica: cdc814f2198148daa1d3c02da8ea09d2, State: Running, Role: LEADER
I20260807 06:37:35.656792 4863 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:35.656824 4882 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc814f2198148daa1d3c02da8ea09d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } }
I20260807 06:37:35.657300 4883 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdc814f2198148daa1d3c02da8ea09d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc814f2198148daa1d3c02da8ea09d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } } }
I20260807 06:37:35.657450 4883 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:35.657500 4884 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdc814f2198148daa1d3c02da8ea09d2. Latest consensus state: current_term: 1 leader_uuid: "cdc814f2198148daa1d3c02da8ea09d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc814f2198148daa1d3c02da8ea09d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45009 } } }
I20260807 06:37:35.657747 4884 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:35.657968 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:35.656585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:35977 (local address 127.0.0.1:45009)
0807 06:37:35.656625 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:35.656628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:35.656829 (+ 201us) server_negotiation.cc:407] Connection header received
0807 06:37:35.656933 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:35.656933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:35.656938 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:35.656979 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:35.657726 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:35.657727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:35.657727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:35.657731 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:35.657744 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:35.657749 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:35.657806 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:35.657895 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 06:37:35.657909 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 06:37:35.658783 4907 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:35.658850 4907 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:35.658900 4901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:35.659053 4901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:35.660632 4901 catalog_manager.cc:1347] Generated new cluster ID: 92aaaab9099c4645a05b1538decf3daa
I20260807 06:37:35.660686 4901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:35.672580 4901 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:35.672782 4901 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:35.682874 4901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdc814f2198148daa1d3c02da8ea09d2: Generated new TSK 0
I20260807 06:37:35.683079 4901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:35.690085 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4764
2026-08-07T06:37:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:35 dist-test-slave-rwrg krb5kdc[4921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:35 dist-test-slave-rwrg krb5kdc[4921](info): set up 2 sockets
Aug 07 06:37:35 dist-test-slave-rwrg krb5kdc[4921](info): commencing operation
krb5kdc: starting...
W20260807 06:37:36.971365 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:37 dist-test-slave-rwrg krb5kdc[4921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:37.095535 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:37.176921 4938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:37.177131 4938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:37.177155 4938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:37.178289 4938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:37.178334 4938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:37.178349 4938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:37.178362 4938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:37.178375 4938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:37.179881 4938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:37.180145 4938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:37.180387 4938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:37.185245 4949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:37.185416 4938 server_base.cc:1034] running on GCE node
W20260807 06:37:37.185336 4946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:37.185245 4945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:37.185742 4938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:37.185984 4938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:37.187188 4938 hybrid_clock.cc:648] HybridClock initialized: now 1786084657187188 us; error 37 us; skew 500 ppm
Aug 07 06:37:37 dist-test-slave-rwrg krb5kdc[4921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:37.189822 4938 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:37.190407 4938 webserver.cc:460] Webserver started at http://127.0.0.1:39237/ using document root <none> and password file <none>
I20260807 06:37:37.190636 4938 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:37.190708 4938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:37.190860 4938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:37.191934 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "d717ba5e278041e0ad5a8a006619cf17"
format_stamp: "Formatted at 2026-08-07 06:37:37 on dist-test-slave-rwrg"
server_key: "a66ac2b38da682b34103870abdf736e1"
server_key_iv: "351d7dd1c5cfd635aeda464a3437906c"
server_key_version: "encryptionkey@0"
I20260807 06:37:37.192315 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "d717ba5e278041e0ad5a8a006619cf17"
format_stamp: "Formatted at 2026-08-07 06:37:37 on dist-test-slave-rwrg"
server_key: "a66ac2b38da682b34103870abdf736e1"
server_key_iv: "351d7dd1c5cfd635aeda464a3437906c"
server_key_version: "encryptionkey@0"
I20260807 06:37:37.193763 4938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:37.194557 4960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:37.194746 4938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:37:37.194850 4938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "d717ba5e278041e0ad5a8a006619cf17"
format_stamp: "Formatted at 2026-08-07 06:37:37 on dist-test-slave-rwrg"
server_key: "a66ac2b38da682b34103870abdf736e1"
server_key_iv: "351d7dd1c5cfd635aeda464a3437906c"
server_key_version: "encryptionkey@0"
I20260807 06:37:37.194947 4938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:37.207415 4938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:37.209527 4938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:37.209698 4938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:37.213925 4938 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41713
I20260807 06:37:37.213975 5055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41713 every 8 connection(s)
I20260807 06:37:37.214320 4938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:37.215322 5058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:37.217643 5058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17: Bootstrap starting.
I20260807 06:37:37.218240 5058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:37.218528 5058 log.cc:824] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:37.219226 5058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17: No bootstrap required, opened a new log
I20260807 06:37:37.220801 5058 raft_consensus.cc:348] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717ba5e278041e0ad5a8a006619cf17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41713 } }
I20260807 06:37:37.220956 5058 raft_consensus.cc:374] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:37.220960 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 4938
I20260807 06:37:37.220999 5058 raft_consensus.cc:729] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d717ba5e278041e0ad5a8a006619cf17, State: Initialized, Role: FOLLOWER
I20260807 06:37:37.221086 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:37.221160 5058 consensus_queue.cc:260] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717ba5e278041e0ad5a8a006619cf17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41713 } }
I20260807 06:37:37.221264 5058 raft_consensus.cc:388] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:37.221308 5058 raft_consensus.cc:482] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:37.221303 3020 external_mini_cluster.cc:1428] Setting key 8c40e899a78ca8996b29ad2097dd1ccb
I20260807 06:37:37.221359 5058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:37.222060 5058 raft_consensus.cc:504] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717ba5e278041e0ad5a8a006619cf17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41713 } }
I20260807 06:37:37.222203 5058 leader_election.cc:302] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d717ba5e278041e0ad5a8a006619cf17; no voters:
I20260807 06:37:37.222407 5058 leader_election.cc:288] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:37.222427 5065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:37.222543 5065 raft_consensus.cc:686] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [term 1 LEADER]: Becoming Leader. State: Replica: d717ba5e278041e0ad5a8a006619cf17, State: Running, Role: LEADER
I20260807 06:37:37.222707 5058 sys_catalog.cc:564] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:37.222710 5065 consensus_queue.cc:237] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717ba5e278041e0ad5a8a006619cf17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41713 } }
I20260807 06:37:37.223198 5067 sys_catalog.cc:455] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d717ba5e278041e0ad5a8a006619cf17. Latest consensus state: current_term: 1 leader_uuid: "d717ba5e278041e0ad5a8a006619cf17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717ba5e278041e0ad5a8a006619cf17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41713 } } }
I20260807 06:37:37.223292 5067 sys_catalog.cc:458] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:37.223512 5066 sys_catalog.cc:455] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d717ba5e278041e0ad5a8a006619cf17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717ba5e278041e0ad5a8a006619cf17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41713 } } }
I20260807 06:37:37.223891 5066 sys_catalog.cc:458] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17 [sys.catalog]: This master's current role is: LEADER
W20260807 06:37:37.224653 5087 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:37.224714 5087 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:37.224808 5077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:37.224972 5077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:37.226372 5077 catalog_manager.cc:1347] Generated new cluster ID: b4a0530754c341eba974b5683783284a
I20260807 06:37:37.226421 5077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:37:37 dist-test-slave-rwrg krb5kdc[4921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:37:37.234347 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:37.222089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:37889 (local address 127.0.0.1:41713)
0807 06:37:37.222259 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:37.222266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:37.222315 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:37:37.222494 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:37.222501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:37.222609 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:37.223576 (+ 967us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:37.223717 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:37.225353 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:37.226393 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:37.227015 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:37.227141 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:37.230048 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:37.230052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:37.230053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:37.230065 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:37.232608 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:37.233076 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:37.233079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:37.233079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:37.233210 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:37.233485 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:37.233486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:37.233486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:37.233762 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:37.233910 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:37.234050 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 06:37:37.234149 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260807 06:37:37.235600 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:37.237499 5077 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:37.238247 5077 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:37.244390 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:37.237247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:34021 (local address 127.0.0.1:41713)
0807 06:37:37.237281 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:37.237283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:37.237380 (+ 97us) server_negotiation.cc:407] Connection header received
0807 06:37:37.237514 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:37.237515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:37.237524 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:37.237583 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:37.238481 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:37.239140 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:37.240149 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:37.240475 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:37.240547 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:37.241862 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:37.241863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:37.241863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:37.241869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:37.243064 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:37.243441 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:37.243442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:37.243443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:37.243495 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:37.243738 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:37.243739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:37.243740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:37.243847 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:37.244176 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:37.244236 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:37:37.244305 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:37:37.252053 5077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d717ba5e278041e0ad5a8a006619cf17: Generated new TSK 0
I20260807 06:37:37.252240 5077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:37.254102 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 4938
W20260807 06:37:37.265830 4915 connection.cc:535] client connection to 127.0.0.1:41713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41713): Connection reset by peer (error 104)
2026-08-07T06:37:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:37 dist-test-slave-rwrg krb5kdc[5100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:37 dist-test-slave-rwrg krb5kdc[5100](info): set up 2 sockets
Aug 07 06:37:37 dist-test-slave-rwrg krb5kdc[5100](info): commencing operation
krb5kdc: starting...
W20260807 06:37:38.510439 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:38 dist-test-slave-rwrg krb5kdc[5100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:38.633431 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:38.715680 5118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:38.715885 5118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:38.715911 5118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:38.717006 5118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:38.717044 5118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:38.717058 5118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:38.717072 5118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:38.717083 5118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:38.718658 5118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:38.718861 5118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:38.719033 5118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:38.723917 5128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:38.723941 5127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:38.723932 5130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:38.724252 5118 server_base.cc:1034] running on GCE node
I20260807 06:37:38.724583 5118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:38.724889 5118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:38.726071 5118 hybrid_clock.cc:648] HybridClock initialized: now 1786084658726072 us; error 50 us; skew 500 ppm
Aug 07 06:37:38 dist-test-slave-rwrg krb5kdc[5100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084658, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:38.728678 5118 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:38.729122 5118 webserver.cc:460] Webserver started at http://127.0.0.1:42759/ using document root <none> and password file <none>
I20260807 06:37:38.729390 5118 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:38.729460 5118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:38.729635 5118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:38.730696 5118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "3932c49d1e14416fbfc00d208f525779"
format_stamp: "Formatted at 2026-08-07 06:37:38 on dist-test-slave-rwrg"
server_key: "b44a30f8883f52f9733e3ed1f327eba2"
server_key_iv: "f843b0db54bac0efcb8aee22b9479e2b"
server_key_version: "encryptionkey@0"
I20260807 06:37:38.731068 5118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "3932c49d1e14416fbfc00d208f525779"
format_stamp: "Formatted at 2026-08-07 06:37:38 on dist-test-slave-rwrg"
server_key: "b44a30f8883f52f9733e3ed1f327eba2"
server_key_iv: "f843b0db54bac0efcb8aee22b9479e2b"
server_key_version: "encryptionkey@0"
I20260807 06:37:38.732463 5118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:38.733353 5143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:38.733573 5118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:38.733667 5118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "3932c49d1e14416fbfc00d208f525779"
format_stamp: "Formatted at 2026-08-07 06:37:38 on dist-test-slave-rwrg"
server_key: "b44a30f8883f52f9733e3ed1f327eba2"
server_key_iv: "f843b0db54bac0efcb8aee22b9479e2b"
server_key_version: "encryptionkey@0"
I20260807 06:37:38.733752 5118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:38.743314 5118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:38.745396 5118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:38.745568 5118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:38.749678 5118 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44863
I20260807 06:37:38.749735 5231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44863 every 8 connection(s)
I20260807 06:37:38.750243 5118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:38.750957 5233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:38.753396 5233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779: Bootstrap starting.
I20260807 06:37:38.753948 5233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:38.754189 5233 log.cc:824] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:38.754953 5233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779: No bootstrap required, opened a new log
I20260807 06:37:38.756728 5233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3932c49d1e14416fbfc00d208f525779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260807 06:37:38.756870 5233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:38.756896 5233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3932c49d1e14416fbfc00d208f525779, State: Initialized, Role: FOLLOWER
I20260807 06:37:38.757066 5233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3932c49d1e14416fbfc00d208f525779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260807 06:37:38.757154 5233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:38.757180 5233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:38.757223 5233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:38.757759 5233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3932c49d1e14416fbfc00d208f525779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260807 06:37:38.757850 5233 leader_election.cc:302] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3932c49d1e14416fbfc00d208f525779; no voters:
I20260807 06:37:38.758003 5233 leader_election.cc:288] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:38.758064 5238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:38.758263 5238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [term 1 LEADER]: Becoming Leader. State: Replica: 3932c49d1e14416fbfc00d208f525779, State: Running, Role: LEADER
I20260807 06:37:38.758325 5233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:38.758407 5238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3932c49d1e14416fbfc00d208f525779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260807 06:37:38.758726 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5118
I20260807 06:37:38.758812 5241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3932c49d1e14416fbfc00d208f525779. Latest consensus state: current_term: 1 leader_uuid: "3932c49d1e14416fbfc00d208f525779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3932c49d1e14416fbfc00d208f525779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } } }
I20260807 06:37:38.758858 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:38.758891 5241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:38.759028 5239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3932c49d1e14416fbfc00d208f525779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3932c49d1e14416fbfc00d208f525779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } } }
I20260807 06:37:38.759084 5239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:38.759107 3020 external_mini_cluster.cc:1428] Setting key 9e601ad2a21578d3591414fbd90dc188
I20260807 06:37:38.759603 5248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:38.760290 5248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:38.762068 5248 catalog_manager.cc:1347] Generated new cluster ID: f1fced724bcb4612b17777360abbf083
I20260807 06:37:38.762127 5248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:38.765331 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:38.759801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:39037 (local address 127.0.0.1:44863)
0807 06:37:38.759984 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:38.759995 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:38.760111 (+ 116us) server_negotiation.cc:407] Connection header received
0807 06:37:38.760309 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:38.760316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:38.760384 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:38.760789 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:38.760863 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:38.762092 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:38.763155 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:38.763737 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:38.763878 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:38.764614 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:38.764618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:38.764618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:38.764848 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:38.764888 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:38.764903 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:38.765016 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:38.765097 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 06:37:38.765125 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260807 06:37:38.766465 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:38.776093 5248 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:38.776633 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:38.768782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:34637 (local address 127.0.0.1:44863)
0807 06:37:38.768815 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:38.768817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:38.772707 (+ 3890us) server_negotiation.cc:407] Connection header received
0807 06:37:38.772781 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:38.772782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:38.772792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:38.772852 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:38.773045 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:38.773797 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:38.774916 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:38.775237 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:38.775305 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:38.776319 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:38.776321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:38.776321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:38.776326 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:38.776342 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:38.776346 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:38.776415 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:38.776554 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 06:37:38.776566 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 06:37:38.776844 5248 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:38.781250 5248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3932c49d1e14416fbfc00d208f525779: Generated new TSK 0
I20260807 06:37:38.781467 5248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:38.786158 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5118
W20260807 06:37:38.801340 5097 connection.cc:535] client connection to 127.0.0.1:44863 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44863): Connection reset by peer (error 104)
2026-08-07T06:37:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:38 dist-test-slave-rwrg krb5kdc[5279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:38 dist-test-slave-rwrg krb5kdc[5279](info): set up 2 sockets
Aug 07 06:37:38 dist-test-slave-rwrg krb5kdc[5279](info): commencing operation
krb5kdc: starting...
W20260807 06:37:40.055020 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:40 dist-test-slave-rwrg krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:37:40.178856 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35283
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:40.261425 5299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:40.261665 5299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:40.261705 5299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:40.262856 5299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:40.262920 5299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:40.262944 5299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:40.262970 5299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:40.263013 5299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:40.264586 5299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35283
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:40.264868 5299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:40.265098 5299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:40.270087 5310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:40.270097 5312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:40.270300 5299 server_base.cc:1034] running on GCE node
W20260807 06:37:40.270087 5315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:40.270613 5299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:40.270947 5299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:40.272116 5299 hybrid_clock.cc:648] HybridClock initialized: now 1786084660272106 us; error 28 us; skew 500 ppm
Aug 07 06:37:40 dist-test-slave-rwrg krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:40.277341 5299 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:37:40.277840 5299 webserver.cc:460] Webserver started at http://127.0.0.1:40785/ using document root <none> and password file <none>
I20260807 06:37:40.278105 5299 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:40.278172 5299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:40.278313 5299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:40.279352 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "7bb0ee64ddd24fa9b77dde11054697c7"
format_stamp: "Formatted at 2026-08-07 06:37:40 on dist-test-slave-rwrg"
server_key: "ff6d724e2013ed90b005fa35c7f03242"
server_key_iv: "504e6b98df140cd6fbb850c7b7ab155f"
server_key_version: "encryptionkey@0"
I20260807 06:37:40.279713 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "7bb0ee64ddd24fa9b77dde11054697c7"
format_stamp: "Formatted at 2026-08-07 06:37:40 on dist-test-slave-rwrg"
server_key: "ff6d724e2013ed90b005fa35c7f03242"
server_key_iv: "504e6b98df140cd6fbb850c7b7ab155f"
server_key_version: "encryptionkey@0"
I20260807 06:37:40.281129 5299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:40.281880 5327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:40.282124 5299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:40.282233 5299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "7bb0ee64ddd24fa9b77dde11054697c7"
format_stamp: "Formatted at 2026-08-07 06:37:40 on dist-test-slave-rwrg"
server_key: "ff6d724e2013ed90b005fa35c7f03242"
server_key_iv: "504e6b98df140cd6fbb850c7b7ab155f"
server_key_version: "encryptionkey@0"
I20260807 06:37:40.282306 5299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:40.285717 5299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:40.287669 5299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:40.287825 5299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:40.292044 5299 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35283
I20260807 06:37:40.292081 5407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35283 every 8 connection(s)
I20260807 06:37:40.292407 5299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:40.293350 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:40.293672 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5299
I20260807 06:37:40.293792 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:40.293973 3020 external_mini_cluster.cc:1428] Setting key d54758640a39c7ba9a2fd01fedda1868
I20260807 06:37:40.296304 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:40.294536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:60907 (local address 127.0.0.1:35283)
0807 06:37:40.294716 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:40.294722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:40.294747 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:37:40.295121 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:40.295127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:40.295151 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:40.295261 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:40.295871 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:40.295875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:40.295875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:40.295878 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:40.295911 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:40.295920 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:40.295973 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:40.296071 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 06:37:40.296125 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260807 06:37:40.296344 5410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7: Bootstrap starting.
I20260807 06:37:40.297175 5410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:40.297492 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:40.297503 5410 log.cc:824] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:40.298269 5410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7: No bootstrap required, opened a new log
I20260807 06:37:40.300063 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:40.299098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:54415 (local address 127.0.0.1:35283)
0807 06:37:40.299138 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:40.299140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:40.299213 (+ 73us) server_negotiation.cc:407] Connection header received
0807 06:37:40.299267 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:40.299267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:40.299271 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:40.299318 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:40.299837 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:40.299838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:40.299839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:40.299842 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:40.299853 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:40.299856 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:40.299899 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:40.299954 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 06:37:40.299965 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:37:40.300905 5410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35283 } }
I20260807 06:37:40.301047 5410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:40.301071 5410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb0ee64ddd24fa9b77dde11054697c7, State: Initialized, Role: FOLLOWER
I20260807 06:37:40.301192 5410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35283 } }
I20260807 06:37:40.301276 5410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:40.301321 5410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:40.301373 5410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:40.301911 5410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35283 } }
I20260807 06:37:40.302028 5410 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bb0ee64ddd24fa9b77dde11054697c7; no voters:
I20260807 06:37:40.302187 5410 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:40.302233 5422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:40.302428 5422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [term 1 LEADER]: Becoming Leader. State: Replica: 7bb0ee64ddd24fa9b77dde11054697c7, State: Running, Role: LEADER
I20260807 06:37:40.302443 5410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:40.302584 5422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35283 } }
I20260807 06:37:40.303107 5424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35283 } } }
I20260807 06:37:40.303222 5424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:40.303110 5425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bb0ee64ddd24fa9b77dde11054697c7. Latest consensus state: current_term: 1 leader_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb0ee64ddd24fa9b77dde11054697c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35283 } } }
I20260807 06:37:40.303273 5425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:40.303426 5438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:40.304046 5438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:40.308518 5438 catalog_manager.cc:1347] Generated new cluster ID: 1eea3f0a5f48472a8242445646a14c9c
I20260807 06:37:40.308588 5438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:40.331693 5438 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:40.331904 5438 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:40.343205 5438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bb0ee64ddd24fa9b77dde11054697c7: Generated new TSK 0
I20260807 06:37:40.343446 5438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:40.347175 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5299
2026-08-07T06:37:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:40 dist-test-slave-rwrg krb5kdc[5458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:40 dist-test-slave-rwrg krb5kdc[5458](info): set up 2 sockets
Aug 07 06:37:40 dist-test-slave-rwrg krb5kdc[5458](info): commencing operation
krb5kdc: starting...
W20260807 06:37:41.607925 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:41 dist-test-slave-rwrg krb5kdc[5458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:41.731880 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:40907
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:40907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:41.812623 5480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:41.812835 5480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:41.812858 5480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:41.814056 5480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:41.814107 5480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:41.814121 5480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:41.814133 5480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:41.814145 5480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:41.815680 5480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:40907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:41.815933 5480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:41.816213 5480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:41.821420 5493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:41.821436 5489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:41.821444 5490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:41.821467 5480 server_base.cc:1034] running on GCE node
I20260807 06:37:41.821910 5480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:41.822249 5480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:41.823426 5480 hybrid_clock.cc:648] HybridClock initialized: now 1786084661823414 us; error 48 us; skew 500 ppm
Aug 07 06:37:41 dist-test-slave-rwrg krb5kdc[5458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084661, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:41.826063 5480 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:41.826575 5480 webserver.cc:460] Webserver started at http://127.2.243.62:32895/ using document root <none> and password file <none>
I20260807 06:37:41.826772 5480 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:41.826817 5480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:41.826911 5480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:41.827986 5480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "a49a22632e8e4238a39372233ce785fc"
format_stamp: "Formatted at 2026-08-07 06:37:41 on dist-test-slave-rwrg"
server_key: "9f522a2808aaf8efbef977574c90923e"
server_key_iv: "d4207dfc9fe30056c04ce65196df7787"
server_key_version: "encryptionkey@0"
I20260807 06:37:41.828293 5480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "a49a22632e8e4238a39372233ce785fc"
format_stamp: "Formatted at 2026-08-07 06:37:41 on dist-test-slave-rwrg"
server_key: "9f522a2808aaf8efbef977574c90923e"
server_key_iv: "d4207dfc9fe30056c04ce65196df7787"
server_key_version: "encryptionkey@0"
I20260807 06:37:41.829789 5480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 06:37:41.830546 5503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:41.830752 5480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:41.830855 5480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "a49a22632e8e4238a39372233ce785fc"
format_stamp: "Formatted at 2026-08-07 06:37:41 on dist-test-slave-rwrg"
server_key: "9f522a2808aaf8efbef977574c90923e"
server_key_iv: "d4207dfc9fe30056c04ce65196df7787"
server_key_version: "encryptionkey@0"
I20260807 06:37:41.830914 5480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:41.843964 5480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:41.845976 5480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:41.846092 5480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:41.850023 5480 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:40907
I20260807 06:37:41.850076 5587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:40907 every 8 connection(s)
I20260807 06:37:41.850420 5480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:41.851359 5591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:41.853754 5591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc: Bootstrap starting.
I20260807 06:37:41.854341 5591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:41.854629 5591 log.cc:824] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:41.855342 5591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc: No bootstrap required, opened a new log
I20260807 06:37:41.856974 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5480
I20260807 06:37:41.857070 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:41.857275 3020 external_mini_cluster.cc:1428] Setting key b57800022280d2c594d35d7d66bab814
I20260807 06:37:41.857191 5591 raft_consensus.cc:348] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49a22632e8e4238a39372233ce785fc" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40907 } }
I20260807 06:37:41.857347 5591 raft_consensus.cc:374] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:41.857396 5591 raft_consensus.cc:729] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a49a22632e8e4238a39372233ce785fc, State: Initialized, Role: FOLLOWER
I20260807 06:37:41.857551 5591 consensus_queue.cc:260] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49a22632e8e4238a39372233ce785fc" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40907 } }
I20260807 06:37:41.857638 5591 raft_consensus.cc:388] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:41.857676 5591 raft_consensus.cc:482] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:41.857784 5591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:41.858642 5591 raft_consensus.cc:504] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49a22632e8e4238a39372233ce785fc" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40907 } }
I20260807 06:37:41.858778 5591 leader_election.cc:302] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a49a22632e8e4238a39372233ce785fc; no voters:
I20260807 06:37:41.858992 5591 leader_election.cc:288] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:41.859017 5597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:41.859136 5597 raft_consensus.cc:686] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [term 1 LEADER]: Becoming Leader. State: Replica: a49a22632e8e4238a39372233ce785fc, State: Running, Role: LEADER
I20260807 06:37:41.859277 5591 sys_catalog.cc:564] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:41.859289 5597 consensus_queue.cc:237] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49a22632e8e4238a39372233ce785fc" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40907 } }
I20260807 06:37:41.859876 5598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a49a22632e8e4238a39372233ce785fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49a22632e8e4238a39372233ce785fc" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40907 } } }
I20260807 06:37:41.859979 5598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:41.859997 5601 sys_catalog.cc:455] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a49a22632e8e4238a39372233ce785fc. Latest consensus state: current_term: 1 leader_uuid: "a49a22632e8e4238a39372233ce785fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49a22632e8e4238a39372233ce785fc" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 40907 } } }
I20260807 06:37:41.860152 5601 sys_catalog.cc:458] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:41.860635 5612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:41.861249 5612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:41.862811 5612 catalog_manager.cc:1347] Generated new cluster ID: ec6f17cefdff42c2a2e0a9988d5171ed
I20260807 06:37:41.862865 5612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 06:37:41 dist-test-slave-rwrg krb5kdc[5458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:41.869325 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:41.857728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47367 (local address 127.2.243.62:40907)
0807 06:37:41.858094 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:41.858102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:41.858128 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:37:41.858228 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:41.858233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:41.858274 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:41.858413 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:41.859197 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:41.860647 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:41.861696 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:41.862310 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:41.862426 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:41.865352 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:41.865356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:41.865356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:41.865368 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:41.867374 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:41.867882 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:41.867885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:41.867886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:41.868008 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:41.868268 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:41.868269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:41.868269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:41.868579 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:41.868744 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:41.868889 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 06:37:41.869114 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":120,"threads_started":1}
I20260807 06:37:41.870355 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:41.879290 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:41.871829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47551 (local address 127.2.243.62:40907)
0807 06:37:41.871862 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:41.871864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:41.871873 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:41.871924 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:41.871924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:41.871933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:41.871985 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:41.872693 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:41.873419 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:41.874453 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:41.874808 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:41.874872 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:41.876326 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:41.876327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:41.876328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:41.876335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:41.878069 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:41.878491 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:41.878492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:41.878492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:41.878547 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:41.878789 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:41.878790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:41.878790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:41.878903 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:41.879007 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:41.879145 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 06:37:41.879204 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 06:37:41.905956 5612 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:41.906533 5612 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:41.911384 5612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a49a22632e8e4238a39372233ce785fc: Generated new TSK 0
I20260807 06:37:41.911629 5612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:41.914451 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5480
W20260807 06:37:41.928129 5455 connection.cc:535] client connection to 127.2.243.62:40907 recv error: Network error: failed to read from TLS socket (remote: 127.2.243.62:40907): Connection reset by peer (error 104)
2026-08-07T06:37:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:41 dist-test-slave-rwrg krb5kdc[5636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:41 dist-test-slave-rwrg krb5kdc[5636](info): set up 2 sockets
Aug 07 06:37:41 dist-test-slave-rwrg krb5kdc[5636](info): commencing operation
krb5kdc: starting...
W20260807 06:37:43.188941 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:43 dist-test-slave-rwrg krb5kdc[5636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:43.315944 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:46803
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:46803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:43.400074 5654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:43.400381 5654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:43.400426 5654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:43.401681 5654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:43.401770 5654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:43.401798 5654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:43.401846 5654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:43.401868 5654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:43.403425 5654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:46803
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:43.403745 5654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:43.403981 5654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:43.409024 5666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:43.409111 5673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:43.409204 5664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:43.409564 5654 server_base.cc:1034] running on GCE node
I20260807 06:37:43.409802 5654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:43.410051 5654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:43.411195 5654 hybrid_clock.cc:648] HybridClock initialized: now 1786084663411162 us; error 52 us; skew 500 ppm
Aug 07 06:37:43 dist-test-slave-rwrg krb5kdc[5636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084663, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:43.413897 5654 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:43.414294 5654 webserver.cc:460] Webserver started at http://127.2.243.62:39981/ using document root <none> and password file <none>
I20260807 06:37:43.414538 5654 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:43.414615 5654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:43.414801 5654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:43.415879 5654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1"
format_stamp: "Formatted at 2026-08-07 06:37:43 on dist-test-slave-rwrg"
server_key: "250769b1756481291a7db63e690ef7f4"
server_key_iv: "6d0da48671758a3d23c7c4476c127d67"
server_key_version: "encryptionkey@0"
I20260807 06:37:43.416266 5654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1"
format_stamp: "Formatted at 2026-08-07 06:37:43 on dist-test-slave-rwrg"
server_key: "250769b1756481291a7db63e690ef7f4"
server_key_iv: "6d0da48671758a3d23c7c4476c127d67"
server_key_version: "encryptionkey@0"
I20260807 06:37:43.417793 5654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:43.418618 5681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:43.418807 5654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:43.418920 5654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1"
format_stamp: "Formatted at 2026-08-07 06:37:43 on dist-test-slave-rwrg"
server_key: "250769b1756481291a7db63e690ef7f4"
server_key_iv: "6d0da48671758a3d23c7c4476c127d67"
server_key_version: "encryptionkey@0"
I20260807 06:37:43.419005 5654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:43.450047 5654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:43.452638 5654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:43.452827 5654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:43.456970 5654 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:46803
I20260807 06:37:43.457043 5753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:46803 every 8 connection(s)
I20260807 06:37:43.457441 5654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:43.458407 5754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:43.461077 5754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1: Bootstrap starting.
I20260807 06:37:43.461499 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5654
I20260807 06:37:43.461637 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:43.461879 3020 external_mini_cluster.cc:1428] Setting key 0f2d439b5f4eab0330579c144324ddde
I20260807 06:37:43.461882 5754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:43.462298 5754 log.cc:824] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:43.463191 5754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1: No bootstrap required, opened a new log
I20260807 06:37:43.465734 5754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46803 } }
I20260807 06:37:43.465919 5754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:43.465955 5754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aee6e3612fb4534b3cd63c9f36c2ae1, State: Initialized, Role: FOLLOWER
I20260807 06:37:43.466112 5754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46803 } }
I20260807 06:37:43.466207 5754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:43.466233 5754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:43.466317 5754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:43.467134 5754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46803 } }
I20260807 06:37:43.467280 5754 leader_election.cc:302] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aee6e3612fb4534b3cd63c9f36c2ae1; no voters:
I20260807 06:37:43.467490 5754 leader_election.cc:288] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:43.467533 5761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:43.467795 5761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [term 1 LEADER]: Becoming Leader. State: Replica: 4aee6e3612fb4534b3cd63c9f36c2ae1, State: Running, Role: LEADER
I20260807 06:37:43.467877 5754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:43.467903 5761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46803 } }
I20260807 06:37:43.468281 5761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aee6e3612fb4534b3cd63c9f36c2ae1. Latest consensus state: current_term: 1 leader_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46803 } } }
I20260807 06:37:43.468362 5761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:43.468571 5759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:43.462605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45087 (local address 127.2.243.62:46803)
0807 06:37:43.462801 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:43.462811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:43.462849 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:37:43.463362 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:43.463370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:43.463419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:43.463541 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:43.463852 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:43.464815 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:43.466113 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:43.466653 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:43.466742 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:43.467712 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:43.467717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:43.467718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:43.467951 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:43.467993 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:43.468008 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:43.468097 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:43.468277 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 06:37:43.468309 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260807 06:37:43.468559 5763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aee6e3612fb4534b3cd63c9f36c2ae1" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 46803 } } }
I20260807 06:37:43.468660 5763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:43.468796 5766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:43.469415 5766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:43.470181 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:43.471144 5766 catalog_manager.cc:1347] Generated new cluster ID: 2a057e2d035541e78957ec192f5a8f0a
I20260807 06:37:43.471204 5766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:43.479509 5766 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:43.480370 5766 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:43.480849 5759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:43.476978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50583 (local address 127.2.243.62:46803)
0807 06:37:43.477046 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:43.477048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:43.477175 (+ 127us) server_negotiation.cc:407] Connection header received
0807 06:37:43.477255 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:43.477255 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:43.477265 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:43.477364 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:43.477524 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:43.478320 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:43.479352 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:43.479597 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:43.479688 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:43.480478 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:43.480480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:43.480480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:43.480485 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:43.480540 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:43.480546 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:43.480608 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:43.480750 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 06:37:43.480762 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":277,"server-negotiator.queue_time_us":44}
I20260807 06:37:43.484395 5766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aee6e3612fb4534b3cd63c9f36c2ae1: Generated new TSK 0
I20260807 06:37:43.484613 5766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:43.488394 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5654
W20260807 06:37:43.501958 5634 connection.cc:535] client connection to 127.2.243.62:46803 recv error: Network error: failed to read from TLS socket (remote: 127.2.243.62:46803): Connection reset by peer (error 104)
2026-08-07T06:37:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:43 dist-test-slave-rwrg krb5kdc[5798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:43 dist-test-slave-rwrg krb5kdc[5798](info): set up 2 sockets
Aug 07 06:37:43 dist-test-slave-rwrg krb5kdc[5798](info): commencing operation
krb5kdc: starting...
W20260807 06:37:44.849886 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:44 dist-test-slave-rwrg krb5kdc[5798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:44.973827 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:44781
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:44781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:45.057374 5819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:45.057593 5819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:45.057617 5819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:45.058838 5819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:45.058882 5819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:45.058897 5819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:45.058908 5819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:45.058920 5819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:45.060446 5819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:44781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:44781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:45.060760 5819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:45.061010 5819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:45.066392 5827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:45.066407 5831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:45.066407 5826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:45.066723 5819 server_base.cc:1034] running on GCE node
I20260807 06:37:45.066974 5819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:45.067265 5819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:45.068437 5819 hybrid_clock.cc:648] HybridClock initialized: now 1786084665068423 us; error 54 us; skew 500 ppm
Aug 07 06:37:45 dist-test-slave-rwrg krb5kdc[5798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084665, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:45.071022 5819 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:45.071497 5819 webserver.cc:460] Webserver started at http://127.2.243.62:38897/ using document root <none> and password file <none>
I20260807 06:37:45.071755 5819 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:45.071802 5819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:45.071971 5819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:45.073163 5819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "bd738c5bf49b406dbbb59162f65315ca"
format_stamp: "Formatted at 2026-08-07 06:37:45 on dist-test-slave-rwrg"
server_key: "fb4a00a9fcdda925b87ec0f3ec3f38ab"
server_key_iv: "d422dfa03f82ae995982b34eb13a1d59"
server_key_version: "encryptionkey@0"
I20260807 06:37:45.073536 5819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "bd738c5bf49b406dbbb59162f65315ca"
format_stamp: "Formatted at 2026-08-07 06:37:45 on dist-test-slave-rwrg"
server_key: "fb4a00a9fcdda925b87ec0f3ec3f38ab"
server_key_iv: "d422dfa03f82ae995982b34eb13a1d59"
server_key_version: "encryptionkey@0"
I20260807 06:37:45.074963 5819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:45.075858 5845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:45.076138 5819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:45.076265 5819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "bd738c5bf49b406dbbb59162f65315ca"
format_stamp: "Formatted at 2026-08-07 06:37:45 on dist-test-slave-rwrg"
server_key: "fb4a00a9fcdda925b87ec0f3ec3f38ab"
server_key_iv: "d422dfa03f82ae995982b34eb13a1d59"
server_key_version: "encryptionkey@0"
I20260807 06:37:45.076349 5819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:45.079945 5819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:45.082082 5819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:45.082254 5819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:45.086741 5819 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:44781
I20260807 06:37:45.086797 5918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:44781 every 8 connection(s)
I20260807 06:37:45.087147 5819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:45.088169 5920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:45.089275 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5819
I20260807 06:37:45.089395 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:45.089612 3020 external_mini_cluster.cc:1428] Setting key d1602a83d6f7830f9254ead9c6151281
I20260807 06:37:45.091367 5920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca: Bootstrap starting.
I20260807 06:37:45.092162 5920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:45.092442 5925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:45.090302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40859 (local address 127.2.243.62:44781)
0807 06:37:45.090530 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:45.090540 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:45.090577 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:37:45.091110 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:45.091119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:45.091154 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:45.091319 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:45.091991 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:45.091994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:45.091994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:45.091999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:45.092039 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:45.092054 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:45.092162 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:45.092198 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 06:37:45.092239 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260807 06:37:45.092610 5920 log.cc:824] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:45.093497 5920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca: No bootstrap required, opened a new log
I20260807 06:37:45.093672 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:45.095878 5920 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd738c5bf49b406dbbb59162f65315ca" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 44781 } }
I20260807 06:37:45.096112 5920 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:45.096284 5925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:45.095049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39389 (local address 127.2.243.62:44781)
0807 06:37:45.095090 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:45.095093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:45.095188 (+ 95us) server_negotiation.cc:407] Connection header received
0807 06:37:45.095232 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:45.095232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:45.095238 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:45.095305 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:45.095916 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:45.095917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:45.095918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:45.095922 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:45.095934 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:45.095938 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:45.095992 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:45.096045 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 06:37:45.096058 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 06:37:45.096216 5920 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd738c5bf49b406dbbb59162f65315ca, State: Initialized, Role: FOLLOWER
I20260807 06:37:45.096539 5920 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd738c5bf49b406dbbb59162f65315ca" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 44781 } }
I20260807 06:37:45.096655 5920 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:45.096696 5920 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:45.096753 5920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:45.097357 5920 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd738c5bf49b406dbbb59162f65315ca" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 44781 } }
I20260807 06:37:45.097453 5920 leader_election.cc:302] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd738c5bf49b406dbbb59162f65315ca; no voters:
I20260807 06:37:45.097661 5920 leader_election.cc:288] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:45.097698 5940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:45.097877 5940 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [term 1 LEADER]: Becoming Leader. State: Replica: bd738c5bf49b406dbbb59162f65315ca, State: Running, Role: LEADER
I20260807 06:37:45.097945 5920 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:45.098016 5940 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd738c5bf49b406dbbb59162f65315ca" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 44781 } }
I20260807 06:37:45.098456 5944 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd738c5bf49b406dbbb59162f65315ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd738c5bf49b406dbbb59162f65315ca" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 44781 } } }
I20260807 06:37:45.098487 5945 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd738c5bf49b406dbbb59162f65315ca. Latest consensus state: current_term: 1 leader_uuid: "bd738c5bf49b406dbbb59162f65315ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd738c5bf49b406dbbb59162f65315ca" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 44781 } } }
I20260807 06:37:45.098676 5944 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:45.098712 5945 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:45.099141 5959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:45.099666 5959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:45.101523 5959 catalog_manager.cc:1347] Generated new cluster ID: 3cf440c7e98746f1b25209a033f290c3
I20260807 06:37:45.101585 5959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:45.117278 5959 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:45.117472 5959 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:45.122812 5959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd738c5bf49b406dbbb59162f65315ca: Generated new TSK 0
I20260807 06:37:45.123006 5959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:45.125079 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5819
2026-08-07T06:37:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:45 dist-test-slave-rwrg krb5kdc[5975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:45 dist-test-slave-rwrg krb5kdc[5975](info): set up 2 sockets
Aug 07 06:37:45 dist-test-slave-rwrg krb5kdc[5975](info): commencing operation
krb5kdc: starting...
W20260807 06:37:46.410243 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:46 dist-test-slave-rwrg krb5kdc[5975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:46.539773 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:41495
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:45825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:41495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:46.624110 5994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:46.624370 5994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:46.624418 5994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:46.625587 5994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:46.625669 5994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:46.625692 5994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:46.625715 5994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:46.625739 5994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:46.627187 5994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:45825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:41495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:41495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:46.627478 5994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:46.627727 5994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:46.632979 6007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:46.633057 5994 server_base.cc:1034] running on GCE node
W20260807 06:37:46.633008 6004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:46.633011 6005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:46.633507 5994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:46.633903 5994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:46.635138 5994 hybrid_clock.cc:648] HybridClock initialized: now 1786084666635134 us; error 38 us; skew 500 ppm
Aug 07 06:37:46 dist-test-slave-rwrg krb5kdc[5975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084666, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:46.641451 5994 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:46.641963 5994 webserver.cc:460] Webserver started at http://127.2.243.62:40889/ using document root <none> and password file <none>
I20260807 06:37:46.642251 5994 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:46.642328 5994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:46.642499 5994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:46.643604 5994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "59641e9031774b7c8b8c8ac3eba4621a"
format_stamp: "Formatted at 2026-08-07 06:37:46 on dist-test-slave-rwrg"
server_key: "c5407ff16a32ac45a9683d5d328c721a"
server_key_iv: "a82ae22b85c1f9750089742138da5164"
server_key_version: "encryptionkey@0"
I20260807 06:37:46.644033 5994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "59641e9031774b7c8b8c8ac3eba4621a"
format_stamp: "Formatted at 2026-08-07 06:37:46 on dist-test-slave-rwrg"
server_key: "c5407ff16a32ac45a9683d5d328c721a"
server_key_iv: "a82ae22b85c1f9750089742138da5164"
server_key_version: "encryptionkey@0"
I20260807 06:37:46.645601 5994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:46.646543 6017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:46.646795 5994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:46.646914 5994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "59641e9031774b7c8b8c8ac3eba4621a"
format_stamp: "Formatted at 2026-08-07 06:37:46 on dist-test-slave-rwrg"
server_key: "c5407ff16a32ac45a9683d5d328c721a"
server_key_iv: "a82ae22b85c1f9750089742138da5164"
server_key_version: "encryptionkey@0"
I20260807 06:37:46.646991 5994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:46.666884 5994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:46.669364 5994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:46.669550 5994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:46.673810 5994 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:41495
I20260807 06:37:46.673858 6093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:41495 every 8 connection(s)
I20260807 06:37:46.674275 5994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:46.675194 6094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:46.675627 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 5994
I20260807 06:37:46.675758 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:46.675931 3020 external_mini_cluster.cc:1428] Setting key ef6a55db4018866f8342177718a65830
I20260807 06:37:46.678457 6094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a: Bootstrap starting.
I20260807 06:37:46.679195 6094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:46.679490 6094 log.cc:824] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:46.680215 6094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a: No bootstrap required, opened a new log
I20260807 06:37:46.682567 6094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59641e9031774b7c8b8c8ac3eba4621a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41495 } }
I20260807 06:37:46.682739 6094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:46.682790 6094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59641e9031774b7c8b8c8ac3eba4621a, State: Initialized, Role: FOLLOWER
I20260807 06:37:46.682924 6094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59641e9031774b7c8b8c8ac3eba4621a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41495 } }
I20260807 06:37:46.683027 6094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:46.683080 6094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:46.683130 6094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 0 FOLLOWER]: Advancing to term 1
Aug 07 06:37:46 dist-test-slave-rwrg krb5kdc[5975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:46.683799 6094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59641e9031774b7c8b8c8ac3eba4621a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41495 } }
I20260807 06:37:46.683912 6094 leader_election.cc:302] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59641e9031774b7c8b8c8ac3eba4621a; no voters:
I20260807 06:37:46.684095 6094 leader_election.cc:288] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:46.684185 6102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:46.684386 6094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:46.684402 6102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [term 1 LEADER]: Becoming Leader. State: Replica: 59641e9031774b7c8b8c8ac3eba4621a, State: Running, Role: LEADER
I20260807 06:37:46.684607 6102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59641e9031774b7c8b8c8ac3eba4621a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41495 } }
I20260807 06:37:46.684981 6103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59641e9031774b7c8b8c8ac3eba4621a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59641e9031774b7c8b8c8ac3eba4621a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41495 } } }
I20260807 06:37:46.685048 6104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59641e9031774b7c8b8c8ac3eba4621a. Latest consensus state: current_term: 1 leader_uuid: "59641e9031774b7c8b8c8ac3eba4621a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59641e9031774b7c8b8c8ac3eba4621a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 41495 } } }
I20260807 06:37:46.685760 6104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:46.686028 6103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a [sys.catalog]: This master's current role is: LEADER
W20260807 06:37:46.686517 6122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:46.686617 6122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:46.686681 6123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:46.686828 6123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:46.687845 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:46.676531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58245 (local address 127.2.243.62:41495)
0807 06:37:46.676731 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:46.676741 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:46.676782 (+ 41us) server_negotiation.cc:407] Connection header received
0807 06:37:46.677318 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:46.677328 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:46.677389 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:46.677537 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:46.678120 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:46.679478 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:46.680398 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:46.681028 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:46.681146 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:46.683829 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:46.683833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:46.683833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:46.683846 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:46.686247 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:46.686644 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:46.686648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:46.686648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:46.686761 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:46.686964 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:46.686965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:46.686965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:46.687208 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:46.687369 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:46.687485 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 06:37:46.687634 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260807 06:37:46.688576 6123 catalog_manager.cc:1347] Generated new cluster ID: 2c2eee932a734b20acc1292d35537d60
I20260807 06:37:46.688629 6123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:46.688861 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:46.696729 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:46.690206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43343 (local address 127.2.243.62:41495)
0807 06:37:46.690242 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:46.690244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:46.690378 (+ 134us) server_negotiation.cc:407] Connection header received
0807 06:37:46.690451 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:46.690452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:46.690462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:46.690518 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:46.691051 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:46.691759 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:46.692470 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:46.692733 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:46.692819 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:46.693979 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:46.693981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:46.693981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:46.693988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:46.695451 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:46.695827 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:46.695829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:46.695829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:46.695888 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:46.696108 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:46.696109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:46.696109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:46.696227 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:46.696331 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:46.696489 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 06:37:46.696607 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:37:46.704174 6123 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:46.704952 6123 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:46.713093 6123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59641e9031774b7c8b8c8ac3eba4621a: Generated new TSK 0
I20260807 06:37:46.713299 6123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:46.718257 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 5994
2026-08-07T06:37:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1596 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:46 dist-test-slave-rwrg krb5kdc[6142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:46 dist-test-slave-rwrg krb5kdc[6142](info): set up 2 sockets
Aug 07 06:37:46 dist-test-slave-rwrg krb5kdc[6142](info): commencing operation
krb5kdc: starting...
W20260807 06:37:47.983286 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:48 dist-test-slave-rwrg krb5kdc[6142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:48.115695 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:32771
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:32771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:48.198482 6161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:48.198757 6161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:48.198801 6161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:48.199991 6161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:48.200075 6161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:48.200100 6161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:48.200122 6161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:48.200150 6161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:48.201786 6161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:32771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:32771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:48.202119 6161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:48.202360 6161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:37:48.207676 6161 server_base.cc:1034] running on GCE node
W20260807 06:37:48.207676 6168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:48.207697 6167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:48.207901 6171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:48.208153 6161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:48.208491 6161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:48.209723 6161 hybrid_clock.cc:648] HybridClock initialized: now 1786084668209708 us; error 31 us; skew 500 ppm
Aug 07 06:37:48 dist-test-slave-rwrg krb5kdc[6142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084668, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:48.212433 6161 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:48.213001 6161 webserver.cc:460] Webserver started at http://127.2.243.62:38681/ using document root <none> and password file <none>
I20260807 06:37:48.213244 6161 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:48.213286 6161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:48.213397 6161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:48.214448 6161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "b5d276d7d34542758ced3dc9a386183a"
format_stamp: "Formatted at 2026-08-07 06:37:48 on dist-test-slave-rwrg"
server_key: "572a2e89f0e59ea843cedabc7252f081"
server_key_iv: "829ab0c25f4328920dfc9eedddc74298"
server_key_version: "encryptionkey@0"
I20260807 06:37:48.214766 6161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "b5d276d7d34542758ced3dc9a386183a"
format_stamp: "Formatted at 2026-08-07 06:37:48 on dist-test-slave-rwrg"
server_key: "572a2e89f0e59ea843cedabc7252f081"
server_key_iv: "829ab0c25f4328920dfc9eedddc74298"
server_key_version: "encryptionkey@0"
I20260807 06:37:48.216117 6161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:48.216946 6181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:48.217141 6161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:37:48.217258 6161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "b5d276d7d34542758ced3dc9a386183a"
format_stamp: "Formatted at 2026-08-07 06:37:48 on dist-test-slave-rwrg"
server_key: "572a2e89f0e59ea843cedabc7252f081"
server_key_iv: "829ab0c25f4328920dfc9eedddc74298"
server_key_version: "encryptionkey@0"
I20260807 06:37:48.217340 6161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:48.227447 6161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:48.229882 6161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:48.230065 6161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:48.234264 6161 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:32771
I20260807 06:37:48.234318 6265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:32771 every 8 connection(s)
I20260807 06:37:48.234686 6161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:48.235625 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:48.238076 6266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a: Bootstrap starting.
I20260807 06:37:48.238718 6266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:48.239035 6266 log.cc:824] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:48.239832 6266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a: No bootstrap required, opened a new log
I20260807 06:37:48.241273 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6161
I20260807 06:37:48.241394 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:48.241598 3020 external_mini_cluster.cc:1428] Setting key 7d0004a3dacfb48269e4f0965878daab
I20260807 06:37:48.241736 6266 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d276d7d34542758ced3dc9a386183a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32771 } }
I20260807 06:37:48.241923 6266 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:48.241971 6266 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d276d7d34542758ced3dc9a386183a, State: Initialized, Role: FOLLOWER
I20260807 06:37:48.242187 6266 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d276d7d34542758ced3dc9a386183a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32771 } }
I20260807 06:37:48.242288 6266 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:48.242327 6266 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:48.242440 6266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:48.243395 6266 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d276d7d34542758ced3dc9a386183a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32771 } }
I20260807 06:37:48.243537 6266 leader_election.cc:302] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5d276d7d34542758ced3dc9a386183a; no voters:
I20260807 06:37:48.243767 6266 leader_election.cc:288] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:48.243789 6273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:48.243903 6273 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [term 1 LEADER]: Becoming Leader. State: Replica: b5d276d7d34542758ced3dc9a386183a, State: Running, Role: LEADER
I20260807 06:37:48.244040 6266 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:48.244060 6273 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d276d7d34542758ced3dc9a386183a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32771 } }
I20260807 06:37:48.244570 6276 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5d276d7d34542758ced3dc9a386183a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d276d7d34542758ced3dc9a386183a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32771 } } }
I20260807 06:37:48.244745 6276 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:48.245224 6290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:48.245549 6279 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5d276d7d34542758ced3dc9a386183a. Latest consensus state: current_term: 1 leader_uuid: "b5d276d7d34542758ced3dc9a386183a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d276d7d34542758ced3dc9a386183a" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 32771 } } }
I20260807 06:37:48.245697 6279 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:48.245800 6290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:48.247538 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:48.242271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35445 (local address 127.2.243.62:32771)
0807 06:37:48.242456 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:48.242464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:48.242488 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:37:48.242626 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:48.242633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:48.242684 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:48.242836 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:48.243033 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:48.244181 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:48.245354 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:48.245985 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:48.246100 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:48.246825 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:48.246829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:48.246829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:48.247051 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:48.247098 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:48.247115 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:48.247180 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:48.247304 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 06:37:48.247372 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260807 06:37:48.247958 6290 catalog_manager.cc:1347] Generated new cluster ID: e38d5626e2ea4034a8ed669918c9cd33
I20260807 06:37:48.248004 6290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:48.248664 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:48.254045 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:48.250196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53345 (local address 127.2.243.62:32771)
0807 06:37:48.250230 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:48.250232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:48.250384 (+ 152us) server_negotiation.cc:407] Connection header received
0807 06:37:48.250446 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:48.250447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:48.250457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:48.250511 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:48.250681 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:48.251374 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:48.252339 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:48.252972 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:48.253056 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:48.253777 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:48.253778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:48.253778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:48.253783 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:48.253798 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:48.253802 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:48.253861 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:48.253925 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 06:37:48.253949 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 06:37:48.256080 6290 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:48.256790 6290 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:48.276821 6290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5d276d7d34542758ced3dc9a386183a: Generated new TSK 0
I20260807 06:37:48.277063 6290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:48.282116 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6161
2026-08-07T06:37:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:48 dist-test-slave-rwrg krb5kdc[6314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:48 dist-test-slave-rwrg krb5kdc[6314](info): set up 2 sockets
Aug 07 06:37:48 dist-test-slave-rwrg krb5kdc[6314](info): commencing operation
krb5kdc: starting...
W20260807 06:37:49.550009 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:49 dist-test-slave-rwrg krb5kdc[6314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:49.680120 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43079
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:41047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:43079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:49.764394 6334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:49.764708 6334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:49.764752 6334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:49.765926 6334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:49.766005 6334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:49.766031 6334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:49.766052 6334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:49.766081 6334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:49.767592 6334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:41047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:43079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:49.767886 6334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:49.768126 6334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:49.773398 6346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:49.773422 6341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:49.773398 6344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:49.774025 6334 server_base.cc:1034] running on GCE node
I20260807 06:37:49.774276 6334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:49.774533 6334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:49.775679 6334 hybrid_clock.cc:648] HybridClock initialized: now 1786084669775668 us; error 29 us; skew 500 ppm
Aug 07 06:37:49 dist-test-slave-rwrg krb5kdc[6314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084669, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:49.778440 6334 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:49.778903 6334 webserver.cc:460] Webserver started at http://127.2.243.62:40685/ using document root <none> and password file <none>
I20260807 06:37:49.779158 6334 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:49.779203 6334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:49.779381 6334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:49.780458 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "f442a0b4cf0c4b37962ace7a6c20065c"
format_stamp: "Formatted at 2026-08-07 06:37:49 on dist-test-slave-rwrg"
server_key: "7b7057c6dddc1396b084628b47bfb935"
server_key_iv: "342fd66337f9902da984edfe49aa8200"
server_key_version: "encryptionkey@0"
I20260807 06:37:49.780905 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "f442a0b4cf0c4b37962ace7a6c20065c"
format_stamp: "Formatted at 2026-08-07 06:37:49 on dist-test-slave-rwrg"
server_key: "7b7057c6dddc1396b084628b47bfb935"
server_key_iv: "342fd66337f9902da984edfe49aa8200"
server_key_version: "encryptionkey@0"
I20260807 06:37:49.782253 6334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:49.783144 6355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:49.783372 6334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:49.783469 6334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "f442a0b4cf0c4b37962ace7a6c20065c"
format_stamp: "Formatted at 2026-08-07 06:37:49 on dist-test-slave-rwrg"
server_key: "7b7057c6dddc1396b084628b47bfb935"
server_key_iv: "342fd66337f9902da984edfe49aa8200"
server_key_version: "encryptionkey@0"
I20260807 06:37:49.783542 6334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:49.787016 6334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:49.789436 6334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:49.789594 6334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:49.793597 6334 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:43079
I20260807 06:37:49.793649 6430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:43079 every 8 connection(s)
I20260807 06:37:49.794029 6334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:49.794976 6431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:49.795372 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6334
I20260807 06:37:49.795459 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:49.795681 3020 external_mini_cluster.cc:1428] Setting key 515a7decf7f639bc9aae48a16d95931f
I20260807 06:37:49.798105 6431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c: Bootstrap starting.
I20260807 06:37:49.798307 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:49.796236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46051 (local address 127.2.243.62:43079)
0807 06:37:49.796434 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:49.796444 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:49.796480 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:37:49.797018 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:49.797027 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:49.797060 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:49.797216 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:49.797872 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:49.797876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:49.797876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:49.797881 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:49.797924 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:49.797939 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:49.798016 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:49.798093 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 06:37:49.798133 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260807 06:37:49.798744 6431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:49.799105 6431 log.cc:824] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:49.799418 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:49.799928 6431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c: No bootstrap required, opened a new log
I20260807 06:37:49.801996 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:49.800941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37085 (local address 127.2.243.62:43079)
0807 06:37:49.800982 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:49.800984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:49.801040 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:37:49.801120 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:49.801121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:49.801125 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:49.801176 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:49.801794 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:49.801795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:49.801795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:37:49.801800 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:49.801813 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:37:49.801817 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:49.801862 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:49.801916 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:37:49.801930 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 06:37:49.802150 6431 raft_consensus.cc:348] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43079 } }
I20260807 06:37:49.802330 6431 raft_consensus.cc:374] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:37:49.802379 6431 raft_consensus.cc:729] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f442a0b4cf0c4b37962ace7a6c20065c, State: Initialized, Role: FOLLOWER
I20260807 06:37:49.802520 6431 consensus_queue.cc:260] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43079 } }
I20260807 06:37:49.802605 6431 raft_consensus.cc:388] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:37:49.802651 6431 raft_consensus.cc:482] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:37:49.802685 6431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:49.803189 6431 raft_consensus.cc:504] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43079 } }
I20260807 06:37:49.803284 6431 leader_election.cc:302] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f442a0b4cf0c4b37962ace7a6c20065c; no voters:
I20260807 06:37:49.803401 6431 leader_election.cc:288] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:37:49.803475 6449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:49.803632 6449 raft_consensus.cc:686] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [term 1 LEADER]: Becoming Leader. State: Replica: f442a0b4cf0c4b37962ace7a6c20065c, State: Running, Role: LEADER
I20260807 06:37:49.803678 6431 sys_catalog.cc:564] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:49.803817 6449 consensus_queue.cc:237] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43079 } }
I20260807 06:37:49.804207 6451 sys_catalog.cc:455] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43079 } } }
I20260807 06:37:49.804313 6451 sys_catalog.cc:458] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:49.804214 6453 sys_catalog.cc:455] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f442a0b4cf0c4b37962ace7a6c20065c. Latest consensus state: current_term: 1 leader_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f442a0b4cf0c4b37962ace7a6c20065c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43079 } } }
I20260807 06:37:49.804438 6453 sys_catalog.cc:458] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c [sys.catalog]: This master's current role is: LEADER
W20260807 06:37:49.805603 6475 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:49.805678 6475 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:49.805725 6464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:49.805879 6464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:49.807430 6464 catalog_manager.cc:1347] Generated new cluster ID: d2cddc2a260e41009d613fb46c0489a6
I20260807 06:37:49.807482 6464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:49.812142 6464 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:49.812310 6464 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:49.815311 6464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f442a0b4cf0c4b37962ace7a6c20065c: Generated new TSK 0
I20260807 06:37:49.815558 6464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:49.819643 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6334
2026-08-07T06:37:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1536 ms)
[----------] 18 tests from AuthTokenIssuingTest (28478 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:49 dist-test-slave-rwrg krb5kdc[6483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:49 dist-test-slave-rwrg krb5kdc[6483](info): set up 2 sockets
Aug 07 06:37:49 dist-test-slave-rwrg krb5kdc[6483](info): commencing operation
krb5kdc: starting...
W20260807 06:37:51.066859 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:51.192308 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43589
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:43589,127.2.243.61:32863,127.2.243.60:45673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:51.274659 6515 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:37:51.274869 6515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:51.274887 6515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:51.274909 6515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:51.276055 6515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:51.276100 6515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:51.276114 6515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:51.276127 6515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:51.276191 6515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:51.277663 6515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:43589,127.2.243.61:32863,127.2.243.60:45673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:43589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:51.277932 6515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:51.278154 6515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:51.283403 6524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:51.283428 6522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:51.283407 6527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:51.283725 6515 server_base.cc:1034] running on GCE node
I20260807 06:37:51.283967 6515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:51.284229 6515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:51.285462 6515 hybrid_clock.cc:648] HybridClock initialized: now 1786084671285427 us; error 53 us; skew 500 ppm
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:51.288290 6515 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:51.288839 6515 webserver.cc:460] Webserver started at http://127.2.243.62:33493/ using document root <none> and password file <none>
I20260807 06:37:51.289104 6515 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:51.289184 6515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:51.289371 6515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:51.290433 6515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "62e62aa2aa274e4b840e8d89cb82fe9c"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "ba3e1fbd5be3757599cbd8aaeee5e9fe"
server_key_iv: "c2a9a47dbac09663c80d9fd02f7f502d"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.290828 6515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "62e62aa2aa274e4b840e8d89cb82fe9c"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "ba3e1fbd5be3757599cbd8aaeee5e9fe"
server_key_iv: "c2a9a47dbac09663c80d9fd02f7f502d"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.292253 6515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:51.293368 6537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:51.293661 6515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:51.293773 6515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "62e62aa2aa274e4b840e8d89cb82fe9c"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "ba3e1fbd5be3757599cbd8aaeee5e9fe"
server_key_iv: "c2a9a47dbac09663c80d9fd02f7f502d"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.293849 6515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:51.307126 6515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:51.309414 6515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:51.309592 6515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:51.313583 6515 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:43589
I20260807 06:37:51.313620 6617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:43589 every 8 connection(s)
I20260807 06:37:51.313974 6515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:51.315047 6618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:51.316735 6618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:51.317399 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6515
I20260807 06:37:51.317500 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:51.317713 3020 external_mini_cluster.cc:1428] Setting key 9014359771c95f5fb3e1f280c4cfc3d4
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.243.61@KRBTEST.COM; defaulting to no policy
I20260807 06:37:51.327881 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.317489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43589 (local address 127.0.0.1:36834)
0807 06:37:51.317686 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.317723 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.318237 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.318576 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.318578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.318821 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.318976 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.318980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.320233 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.320234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.321452 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.321455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.322013 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.322015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.322109 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.322640 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.322651 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.324377 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.326319 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.326325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.326328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.326532 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.326799 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.326799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.326800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.326890 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.327256 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.327261 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.327323 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.327439 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.327692 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":114,"thread_start_us":99,"threads_started":1}
I20260807 06:37:51.327883 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.317708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.2.243.62:43589)
0807 06:37:51.317848 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.317853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.317890 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:37:51.318420 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.318423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.318445 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.318550 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.319094 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.320121 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.321581 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.321879 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.321965 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.324533 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.324540 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.324540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.324547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.326190 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.326636 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.326639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.326640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.326722 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.326962 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.326963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.326963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.327175 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.327278 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.327474 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.327629 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260807 06:37:51.329010 6618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:51.329387 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.329262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45146)
0807 06:37:51.329323 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.329367 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 06:37:51.329600 6539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.61@KRBTEST.COM" created.
I20260807 06:37:51.330070 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.329942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45150)
0807 06:37:51.329998 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.330038 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 06:37:51.330260 6618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:32863: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab.
Entry for principal kudu/127.2.243.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab.
I20260807 06:37:51.347404 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:32863
--webserver_interface=127.2.243.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab
--principal=kudu/127.2.243.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:43589,127.2.243.61:32863,127.2.243.60:45673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:37:51.379552 6618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } attempt: 1
I20260807 06:37:51.380100 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.379956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45164)
0807 06:37:51.380028 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.380071 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 06:37:51.380465 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.380365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45172)
0807 06:37:51.380416 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.380441 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:37:51.380578 6618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:32863: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
W20260807 06:37:51.435604 6627 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:37:51.435853 6627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:51.435883 6627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:51.435920 6627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:51.437119 6627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:51.437188 6627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:51.437214 6627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:51.437243 6627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:51.437263 6627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:51.438743 6627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:43589,127.2.243.61:32863,127.2.243.60:45673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab
--principal=kudu/127.2.243.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:32863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.243.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:51.439009 6627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:51.439236 6627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:51.444332 6639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:51.444389 6641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:51.444335 6637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:51.444635 6627 server_base.cc:1034] running on GCE node
I20260807 06:37:51.444917 6627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:51.445242 6627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:51.446426 6627 hybrid_clock.cc:648] HybridClock initialized: now 1786084671446398 us; error 42 us; skew 500 ppm
I20260807 06:37:51.448716 6618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } attempt: 2
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:51.449146 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.448972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45180)
0807 06:37:51.449061 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.449104 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 06:37:51.449378 6627 init.cc:377] Logged in from keytab as kudu/127.2.243.61@KRBTEST.COM (short username kudu)
I20260807 06:37:51.449584 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.449423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45196)
0807 06:37:51.449508 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.449555 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 06:37:51.449687 6618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:32863: Network error: Client connection negotiation failed: client connection to 127.2.243.61:32863: connect: Connection refused (error 111)
I20260807 06:37:51.449918 6627 webserver.cc:460] Webserver started at http://127.2.243.61:46367/ using document root <none> and password file <none>
I20260807 06:37:51.450196 6627 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:51.450277 6627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:51.450452 6627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:51.451532 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data/instance:
uuid: "e9cb94a5f37a44bc8057e1b736a85cd7"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "a1ea3501654a8d558b96a0c7056537d4"
server_key_iv: "4bf01c634f140ddbe0be2132fab8a4b8"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.451925 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/wal/instance:
uuid: "e9cb94a5f37a44bc8057e1b736a85cd7"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "a1ea3501654a8d558b96a0c7056537d4"
server_key_iv: "4bf01c634f140ddbe0be2132fab8a4b8"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.453464 6627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 06:37:51.454263 6650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:51.454445 6627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 06:37:51.454569 6627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/wal
uuid: "e9cb94a5f37a44bc8057e1b736a85cd7"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "a1ea3501654a8d558b96a0c7056537d4"
server_key_iv: "4bf01c634f140ddbe0be2132fab8a4b8"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.454669 6627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:51.471608 6627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:51.474187 6627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:51.474372 6627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:51.478698 6627 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.61:32863
I20260807 06:37:51.478746 6736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.61:32863 every 8 connection(s)
I20260807 06:37:51.479154 6627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
I20260807 06:37:51.480360 6737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:51.481981 6737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:51.485146 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6627
I20260807 06:37:51.485253 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-1/wal/instance
I20260807 06:37:51.485487 3020 external_mini_cluster.cc:1428] Setting key 8bc01f2b4f60a77fa1bc8aed2f4f1dfe
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:51.492367 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.482828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.2.243.62:43589)
0807 06:37:51.483008 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.483010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.483133 (+ 123us) server_negotiation.cc:407] Connection header received
0807 06:37:51.483594 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.483596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.483609 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.483676 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.484281 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.484879 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.486098 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.486439 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.486518 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.489103 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.489105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.489106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.489114 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.490893 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.491313 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.491315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.491316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.491384 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.491640 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.491641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.491642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.491804 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.491887 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.492147 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.492243 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260807 06:37:51.492376 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.482714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43589 (local address 127.0.0.1:36840)
0807 06:37:51.482980 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.483032 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.483474 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.483755 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.483758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.483995 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.484200 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.484203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.484991 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.484992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.485985 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.485988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.486570 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.486571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.486682 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.487271 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.487282 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.488978 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.490987 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.490994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.490996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.491229 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.491458 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.491459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.491459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.491560 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.491879 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.491883 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.491992 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.492088 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.492207 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260807 06:37:51.493068 6737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.243.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.60@KRBTEST.COM" created.
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:37:51.501748 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.493324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45208)
0807 06:37:51.493376 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.493386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.493422 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.493673 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.493674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.493800 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.493844 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.493846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.494739 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.494740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.495221 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.495223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.495693 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.495694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.495756 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.496157 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.496162 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.498324 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.500683 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.500684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.500684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.500905 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.501134 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.501134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.501134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.501183 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.501518 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.501519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.501562 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.501613 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.501688 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:37:51.501911 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.493405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.2.243.61:32863)
0807 06:37:51.493537 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.493541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.493561 (+ 20us) server_negotiation.cc:407] Connection header received
0807 06:37:51.493575 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.493576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.493586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.493661 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.493922 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.494642 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.495303 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.495592 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.495670 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.498417 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.498426 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.498426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.498433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.500607 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.500993 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.500996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.500996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.501066 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.501244 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.501245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.501245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.501449 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.501534 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.501648 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.501761 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260807 06:37:51.502785 6737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:51.503162 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.503006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41800)
0807 06:37:51.503072 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.503127 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 06:37:51.503309 6652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
I20260807 06:37:51.503780 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.503664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41804)
0807 06:37:51.503718 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.503750 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 06:37:51.503893 6737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:45673: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab.
Entry for principal kudu/127.2.243.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab.
I20260807 06:37:51.515571 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:45673
--webserver_interface=127.2.243.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab
--principal=kudu/127.2.243.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:43589,127.2.243.61:32863,127.2.243.60:45673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:37:51.540853 6618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } attempt: 3
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:37:51.548481 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.541251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45216)
0807 06:37:51.541306 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.541319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.541396 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.541621 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.541622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.541787 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.541854 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.541856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.542562 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.542564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.543154 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.543156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.543590 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.543591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.543689 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.544229 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.544233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.545629 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.547358 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.547359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.547359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.547544 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.547786 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.547787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.547787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.547847 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.548127 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.548127 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.548177 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.548266 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.548383 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 06:37:51.548481 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.541328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.2.243.61:32863)
0807 06:37:51.541402 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.541404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.541417 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:37:51.541449 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.541451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.541463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.541612 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.541946 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.542459 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.543244 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.543492 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.543581 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.545725 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.545727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.545727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.545735 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.547260 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.547639 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.547641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.547641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.547702 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.547923 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.547924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.547925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.548050 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.548134 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.548292 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.548385 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:37:51.549075 6618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:51.549475 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.549361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41818)
0807 06:37:51.549417 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.549448 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 06:37:51.549958 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.549872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41824)
0807 06:37:51.549917 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.549942 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:37:51.550058 6618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:45673: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
I20260807 06:37:51.553120 6737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } attempt: 1
I20260807 06:37:51.553503 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.553353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41838)
0807 06:37:51.553420 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.553463 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:37:51.553889 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.553769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41846)
0807 06:37:51.553825 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.553860 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 06:37:51.554001 6737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:45673: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
I20260807 06:37:51.581190 6618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } attempt: 1
I20260807 06:37:51.581712 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.581564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41862)
0807 06:37:51.581631 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.581684 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 06:37:51.582216 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.582123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41868)
0807 06:37:51.582173 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.582199 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 06:37:51.582334 6618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:45673: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
W20260807 06:37:51.600921 6749 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:37:51.601151 6749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:51.601169 6749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:51.601233 6749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:51.602367 6749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:51.602412 6749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:51.602427 6749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:51.602439 6749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:51.602451 6749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:51.603964 6749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:43589,127.2.243.61:32863,127.2.243.60:45673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab
--principal=kudu/127.2.243.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:45673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.243.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:51.604216 6749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:51.604467 6749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:51.609447 6754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:51.609459 6762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:51.609500 6757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:51.609797 6749 server_base.cc:1034] running on GCE node
I20260807 06:37:51.610044 6749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:51.610345 6749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:51.611498 6749 hybrid_clock.cc:648] HybridClock initialized: now 1786084671611475 us; error 37 us; skew 500 ppm
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:51.614260 6749 init.cc:377] Logged in from keytab as kudu/127.2.243.60@KRBTEST.COM (short username kudu)
I20260807 06:37:51.614753 6749 webserver.cc:460] Webserver started at http://127.2.243.60:40631/ using document root <none> and password file <none>
I20260807 06:37:51.615007 6749 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:51.615083 6749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:51.615239 6749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:51.616348 6749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data/instance:
uuid: "317004fa0b444037b25d5e3b4a95d1d4"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "3976f13c60608b33b121ce4bba5d47e0"
server_key_iv: "fcf01bbd245e011fb034e08f038544ab"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.616809 6749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/wal/instance:
uuid: "317004fa0b444037b25d5e3b4a95d1d4"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "3976f13c60608b33b121ce4bba5d47e0"
server_key_iv: "fcf01bbd245e011fb034e08f038544ab"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.618172 6749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:51.618996 6770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:51.619190 6749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:37:51.619297 6749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/wal
uuid: "317004fa0b444037b25d5e3b4a95d1d4"
format_stamp: "Formatted at 2026-08-07 06:37:51 on dist-test-slave-rwrg"
server_key: "3976f13c60608b33b121ce4bba5d47e0"
server_key_iv: "fcf01bbd245e011fb034e08f038544ab"
server_key_version: "encryptionkey@0"
I20260807 06:37:51.619381 6749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:51.622159 6737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } attempt: 2
I20260807 06:37:51.622634 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.622449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41878)
0807 06:37:51.622552 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.622594 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:37:51.623109 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.622989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41894)
0807 06:37:51.623030 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.623078 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 06:37:51.623237 6737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:45673: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
I20260807 06:37:51.651371 6749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:51.654036 6749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:51.654238 6749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:51.657536 6618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } attempt: 2
I20260807 06:37:51.658013 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.657864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41904)
0807 06:37:51.657937 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.657973 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 06:37:51.658504 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.658363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41906)
0807 06:37:51.658448 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.658478 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 06:37:51.658658 6618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:45673: Network error: Client connection negotiation failed: client connection to 127.2.243.60:45673: connect: Connection refused (error 111)
I20260807 06:37:51.659202 6749 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.60:45673
I20260807 06:37:51.659269 6854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.60:45673 every 8 connection(s)
I20260807 06:37:51.659642 6749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
I20260807 06:37:51.660753 6855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:51.661307 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 6749
I20260807 06:37:51.661408 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786084392481039-3020-0/minicluster-data/master-2/wal/instance
I20260807 06:37:51.661635 3020 external_mini_cluster.cc:1428] Setting key 135cdb164a4aa1199b0be46190776dca
I20260807 06:37:51.662914 6855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:37:51.673489 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.663731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.2.243.62:43589)
0807 06:37:51.663799 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.663801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.663971 (+ 170us) server_negotiation.cc:407] Connection header received
0807 06:37:51.664034 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.664035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.664044 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.664147 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.664702 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.665411 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.666698 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.666976 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.667079 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.669928 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.669929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.669930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.669939 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.671856 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.672424 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.672425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.672426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.672514 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.672815 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.672816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.672816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.672939 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.673024 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.673309 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.673394 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:37:51.673724 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.662282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41910 (local address 127.2.243.60:45673)
0807 06:37:51.662483 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.662492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.662521 (+ 29us) server_negotiation.cc:407] Connection header received
0807 06:37:51.662943 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.662948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.663009 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.663134 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.664133 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.665464 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.666605 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.667113 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.667207 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.670371 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.670379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.670379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.670387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.672121 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.672607 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.672610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.672610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.672725 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.672951 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.672952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.672952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.673167 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.673320 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.673413 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.673511 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260807 06:37:51.673723 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.663613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:43589 (local address 127.0.0.1:36854)
0807 06:37:51.663828 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.663874 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.663956 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.664177 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.664180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.664414 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.664600 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.664606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.665547 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.665548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.666600 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.666602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.667116 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.667116 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.667211 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.667944 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.667955 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.669818 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.672095 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.672102 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.672104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.672325 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.672607 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.672608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.672609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.672727 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.673019 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.673026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.673168 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.673276 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.673432 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":131,"threads_started":1}
I20260807 06:37:51.674479 6855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:51.682452 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.674854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45230 (local address 127.2.243.61:32863)
0807 06:37:51.674891 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.674893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.674900 (+ 7us) server_negotiation.cc:407] Connection header received
0807 06:37:51.674924 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.674925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.674934 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.674986 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.675324 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.676052 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.677203 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.677540 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.677621 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.679687 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.679689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.679690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.679698 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.681199 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.681594 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.681595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.681596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.681653 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.681882 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.681883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.681884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.682041 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.682127 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.682286 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.682369 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:37:51.682452 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.674766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:32863 (local address 127.0.0.1:45230)
0807 06:37:51.674805 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.674815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.674845 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.675023 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.675024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.675176 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.675243 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.675245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.676152 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.676152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.677086 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.677089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.677629 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.677630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.677685 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.678097 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.678102 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.679578 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.681300 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.681301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.681302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.681472 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.681728 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.681729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.681730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.681793 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.682120 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.682121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.682166 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.682241 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.682345 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 06:37:51.683200 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.674683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.2.243.62:43589)
0807 06:37:51.674727 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.674730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.674739 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:51.674789 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.674790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.674801 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.674857 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.675491 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.676207 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.677366 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.677696 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.677748 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.680493 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.680494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.680495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.680502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.681958 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.682323 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.682324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.682324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.682393 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.682613 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.682615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.682615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.682771 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.682985 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.683028 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.683107 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:37:51.683166 6855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:37:51.690222 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.683434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41914)
0807 06:37:51.683523 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.683540 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.683586 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.683733 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.683734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.683916 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.683949 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.683950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.684913 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.684914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.685433 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.685434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.685814 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.685815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.685869 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.686211 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.686216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.687509 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.689138 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.689138 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.689139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.689368 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.689573 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.689573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.689574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.689627 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.689882 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.689882 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.689919 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.689984 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.690090 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 06:37:51.690263 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.683491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.2.243.60:45673)
0807 06:37:51.683576 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.683578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.683588 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:51.683633 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.683634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.683644 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.683707 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.684005 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.684822 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.685530 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.685732 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.685783 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.687614 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.687615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.687616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.687622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.689039 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.689435 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.689436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.689436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.689487 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.689713 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.689713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.689714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.689811 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.689890 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.690026 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.690139 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":19}
I20260807 06:37:51.691915 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.684179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.2.243.61:32863)
0807 06:37:51.684218 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.684220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.684256 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:37:51.684332 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.684333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.684342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.684396 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.684885 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.685337 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.686242 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.686435 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.686488 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.689121 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.689122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.689122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.689129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.690809 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.691143 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.691144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.691144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.691204 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.691392 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.691393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.691393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.691544 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.691709 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.691750 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.691831 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:37:51.692123 6855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: Bootstrap starting.
I20260807 06:37:51.693209 6855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:51.693513 6855 log.cc:824] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:51.694054 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:51.694428 6855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: No bootstrap required, opened a new log
I20260807 06:37:51.695231 6578 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36870
I20260807 06:37:51.696559 6855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
I20260807 06:37:51.696713 6855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 317004fa0b444037b25d5e3b4a95d1d4, State: Initialized, Role: FOLLOWER
I20260807 06:37:51.696791 6855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
I20260807 06:37:51.697122 6855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:51.697088 6869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.697239 6869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:37:51.699000 6881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:51.699085 6881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:51.699394 6749 master_runner.cc:186] Error getting master registration for 127.2.243.62:43589: OK, instance_id { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" instance_seqno: 1786084671312362 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:37:51.701893 6749 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:37:51.714870 6737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } attempt: 3
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:37:51.723570 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.715234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41930)
0807 06:37:51.715291 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.715301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.715341 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.715588 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.715589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.715776 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.715827 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.715829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.716765 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.716766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.717514 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.717518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.718044 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.718045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.718113 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.718665 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.718671 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.720375 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.722232 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.722233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.722234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.722519 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.722736 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.722737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.722737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.722834 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.723223 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.723225 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.723277 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.723354 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.723457 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 06:37:51.723590 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.715297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41930 (local address 127.2.243.60:45673)
0807 06:37:51.715431 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.715433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.715442 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:51.715467 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.715468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.715478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.715539 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.715932 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.716683 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.717622 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.717931 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.718006 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.720460 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.720462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.720462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.720471 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.722112 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.722593 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.722594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.722594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.722659 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.722938 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.722939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.722940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.723144 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.723217 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.723408 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.723496 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 06:37:51.725817 6737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: Bootstrap starting.
I20260807 06:37:51.726593 6737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:51.726892 6737 log.cc:824] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:51.727671 6737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: No bootstrap required, opened a new log
I20260807 06:37:51.729218 6737 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
I20260807 06:37:51.729387 6737 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cb94a5f37a44bc8057e1b736a85cd7, State: Initialized, Role: FOLLOWER
I20260807 06:37:51.729480 6737 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
I20260807 06:37:51.729786 6885 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.729866 6737 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:51.729883 6885 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:37:51.731794 6902 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:51.731858 6902 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:51.731884 6627 master_runner.cc:186] Error getting master registration for 127.2.243.62:43589: OK, instance_id { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" instance_seqno: 1786084671312362 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:37:51.734131 6627 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:37:51.757786 6618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } attempt: 3
Aug 07 06:37:51 dist-test-slave-rwrg krb5kdc[6483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:37:51.765802 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.758211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:45673 (local address 127.0.0.1:41946)
0807 06:37:51.758286 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:51.758298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:51.758353 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:51.758494 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:51.758495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:51.758690 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:51.758737 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.758739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.759620 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.759621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.760161 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:51.760162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.760657 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.760658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:51.760714 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.761218 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:51.761222 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:51.762835 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:51.764640 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.764641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.764642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.764881 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.765078 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:51.765079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:51.765079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:51.765140 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:51.765408 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:51.765410 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:51.765454 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 06:37:51.765565 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 06:37:51.765674 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 06:37:51.767323 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.758290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.2.243.60:45673)
0807 06:37:51.758338 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.758341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.758350 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:51.758431 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.758432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.758442 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.758501 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.758851 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.759554 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.760266 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.760594 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.760658 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.762913 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.762915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.762916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.762924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.764442 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.764943 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.764944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.764945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.765004 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.765192 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.765193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.765194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.765342 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.765401 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.767141 (+ 1740us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.767228 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 06:37:51.768766 6618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c: Bootstrap starting.
I20260807 06:37:51.769460 6618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:51.769780 6618 log.cc:824] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:51.770661 6618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c: No bootstrap required, opened a new log
I20260807 06:37:51.772070 6618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
I20260807 06:37:51.772215 6618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e62aa2aa274e4b840e8d89cb82fe9c, State: Initialized, Role: FOLLOWER
I20260807 06:37:51.772310 6618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
I20260807 06:37:51.772691 6618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:51.772727 6578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 06:37:51.772769 6578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:51.773644 6578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
I20260807 06:37:51.773909 6578 leader_election.cc:288] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [CANDIDATE]: Term 1 election: Requested vote from peers e9cb94a5f37a44bc8057e1b736a85cd7 (127.2.243.61:32863), 317004fa0b444037b25d5e3b4a95d1d4 (127.2.243.60:45673)
I20260807 06:37:51.774466 6815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "317004fa0b444037b25d5e3b4a95d1d4"
I20260807 06:37:51.774626 6815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:51.775552 6815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e62aa2aa274e4b840e8d89cb82fe9c in term 1.
I20260807 06:37:51.776197 6695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7"
I20260807 06:37:51.776335 6695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:51.776759 6539 leader_election.cc:302] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 317004fa0b444037b25d5e3b4a95d1d4, 62e62aa2aa274e4b840e8d89cb82fe9c; no voters:
I20260807 06:37:51.779523 6695 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e62aa2aa274e4b840e8d89cb82fe9c in term 1.
I20260807 06:37:51.780582 6930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:51.780767 6930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [term 1 LEADER]: Becoming Leader. State: Replica: 62e62aa2aa274e4b840e8d89cb82fe9c, State: Running, Role: LEADER
I20260807 06:37:51.780860 6930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } }
W20260807 06:37:51.774735 6924 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 06:37:51.772658 6907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.781913 6515 master_runner.cc:263] No leader master found from master 62e62aa2aa274e4b840e8d89cb82fe9c
I20260807 06:37:51.781958 6515 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:37:51.781472 6907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:51.781282 6930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e62aa2aa274e4b840e8d89cb82fe9c. Latest consensus state: current_term: 1 leader_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
W20260807 06:37:51.781293 6924 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:51.783661 6930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:51.793462 6952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:51.793701 6952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:51.801466 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.782401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.2.243.62:43589)
0807 06:37:51.782442 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.782444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.783380 (+ 936us) server_negotiation.cc:407] Connection header received
0807 06:37:51.783480 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.783481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.783504 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.783671 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.784682 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.785440 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.790313 (+ 4873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.790619 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.790665 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.797377 (+ 6712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.797379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.797379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.797387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.799272 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.800277 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.800278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.800279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.800340 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.800848 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.800849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.800849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.800999 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.801096 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.801288 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.801369 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 06:37:51.805745 6695 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [term 1 FOLLOWER]: Refusing update from remote peer 62e62aa2aa274e4b840e8d89cb82fe9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:51.806056 6815 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [term 1 FOLLOWER]: Refusing update from remote peer 62e62aa2aa274e4b840e8d89cb82fe9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:51.806592 6930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:51.806893 6930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:51.808146 6869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e62aa2aa274e4b840e8d89cb82fe9c. Latest consensus state: current_term: 1 leader_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.808274 6869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:51.809218 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.782794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45252 (local address 127.2.243.61:32863)
0807 06:37:51.782832 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.782833 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.786586 (+ 3753us) server_negotiation.cc:407] Connection header received
0807 06:37:51.786681 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.786682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.786692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.786722 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.788456 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.789178 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.793836 (+ 4658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.794167 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.794210 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.802608 (+ 8398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.802609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.802610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.802619 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.804293 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.807099 (+ 2806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.807101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.807101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.807165 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.807816 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.807817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.807817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.807961 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.808055 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.809033 (+ 978us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.809123 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:51.809384 6885 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e62aa2aa274e4b840e8d89cb82fe9c. Latest consensus state: current_term: 1 leader_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.809479 6885 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:37:51.810381 6547 tablet.cc:2369] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:37:51.810535 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:51.783114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.2.243.60:45673)
0807 06:37:51.783151 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.783153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.787571 (+ 4418us) server_negotiation.cc:407] Connection header received
0807 06:37:51.787668 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.787669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.787680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.787710 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.791225 (+ 3515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.791949 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.795195 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:51.795520 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:51.795562 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:51.804753 (+ 9191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:51.804755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:51.804756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:51.804764 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:51.806398 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.807674 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.807675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.807675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.807732 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:51.808645 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:51.808646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:51.808646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:51.808798 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:51.808879 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:51.810361 (+ 1482us) server_negotiation.cc:299] Negotiation successful
0807 06:37:51.810449 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 06:37:51.818274 6930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.818369 6930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:51.818679 6953 mvcc.cc:205] Tried to move back new op lower bound from 7315802815712436224 to 7315802815615152128. Current Snapshot: MvccSnapshot[applied={T|T < 7315802815712436224}]
I20260807 06:37:51.819293 6869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.819373 6869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:51.819994 6907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.820075 6907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:51.821388 6885 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e62aa2aa274e4b840e8d89cb82fe9c" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 43589 } } peers { permanent_uuid: "e9cb94a5f37a44bc8057e1b736a85cd7" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 32863 } } peers { permanent_uuid: "317004fa0b444037b25d5e3b4a95d1d4" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 45673 } } }
I20260807 06:37:51.821472 6885 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:51.821933 6959 mvcc.cc:205] Tried to move back new op lower bound from 7315802815712436224 to 7315802815615152128. Current Snapshot: MvccSnapshot[applied={T|T < 7315802815712436224}]
I20260807 06:37:51.824937 6952 catalog_manager.cc:1347] Generated new cluster ID: 0f8ca270b0064d3c81d1fa88847cd0a5
I20260807 06:37:51.824997 6952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:51.844660 6952 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:51.845395 6952 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:51.850627 6952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62e62aa2aa274e4b840e8d89cb82fe9c: Generated new TSK 0
I20260807 06:37:51.850821 6952 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:37:51.883713 6621 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.882616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.2.243.62:43589)
0807 06:37:51.882662 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.882664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.882996 (+ 332us) server_negotiation.cc:407] Connection header received
0807 06:37:51.883109 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.883111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.883122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.883155 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.883639 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36884: BlockingRecv error: recv got EOF from 127.0.0.1:36884 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 06:37:51.885129 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.884167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.2.243.61:32863)
0807 06:37:51.884212 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.884214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.884602 (+ 388us) server_negotiation.cc:407] Connection header received
0807 06:37:51.884704 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.884705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.884716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.884745 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.885071 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45266: BlockingRecv error: recv got EOF from 127.0.0.1:45266 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 06:37:51.885967 6863 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.884427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41958 (local address 127.2.243.60:45673)
0807 06:37:51.884470 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.884473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.885399 (+ 926us) server_negotiation.cc:407] Connection header received
0807 06:37:51.885505 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.885506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.885519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.885572 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.885908 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41958: BlockingRecv error: recv got EOF from 127.0.0.1:41958 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 06:37:51.888325 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.887582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.2.243.61:32863)
0807 06:37:51.887620 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.887622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.887813 (+ 191us) server_negotiation.cc:407] Connection header received
0807 06:37:51.887913 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.887914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.887929 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.887958 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.888283 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45268: BlockingRecv error: recv got EOF from 127.0.0.1:45268 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:37:51.891436 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.890668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.2.243.61:32863)
0807 06:37:51.890703 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.890704 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.890924 (+ 220us) server_negotiation.cc:407] Connection header received
0807 06:37:51.891019 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.891019 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.891029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.891059 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.891393 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45270: BlockingRecv error: recv got EOF from 127.0.0.1:45270 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:37:51.895781 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.894993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.2.243.61:32863)
0807 06:37:51.895027 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.895029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.895248 (+ 219us) server_negotiation.cc:407] Connection header received
0807 06:37:51.895340 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.895341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.895350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.895379 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.895738 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45286: BlockingRecv error: recv got EOF from 127.0.0.1:45286 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 06:37:51.901850 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.901031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.2.243.61:32863)
0807 06:37:51.901069 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.901071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.901307 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:37:51.901414 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.901415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.901425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.901454 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.901802 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45296: BlockingRecv error: recv got EOF from 127.0.0.1:45296 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 06:37:51.911964 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.911200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.2.243.61:32863)
0807 06:37:51.911242 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.911244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.911382 (+ 138us) server_negotiation.cc:407] Connection header received
0807 06:37:51.911433 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.911434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.911445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.911521 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.911901 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45312: BlockingRecv error: recv got EOF from 127.0.0.1:45312 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:37:51.930056 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.929272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45316 (local address 127.2.243.61:32863)
0807 06:37:51.929352 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.929354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.929405 (+ 51us) server_negotiation.cc:407] Connection header received
0807 06:37:51.929463 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.929464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.929477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.929566 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.929993 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45316: BlockingRecv error: recv got EOF from 127.0.0.1:45316 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 06:37:51.964248 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:51.963398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.2.243.61:32863)
0807 06:37:51.963509 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:51.963512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:51.963640 (+ 128us) server_negotiation.cc:407] Connection header received
0807 06:37:51.963731 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:51.963732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:51.963745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:51.963820 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:51.964195 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45328: BlockingRecv error: recv got EOF from 127.0.0.1:45328 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 06:37:52.030684 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:52.029754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.2.243.61:32863)
0807 06:37:52.029850 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.029852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.029997 (+ 145us) server_negotiation.cc:407] Connection header received
0807 06:37:52.030122 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.030123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.030136 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.030214 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:52.030620 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45336: BlockingRecv error: recv got EOF from 127.0.0.1:45336 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 06:37:52.160955 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:52.160020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.2.243.61:32863)
0807 06:37:52.160101 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.160104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.160225 (+ 121us) server_negotiation.cc:407] Connection header received
0807 06:37:52.160335 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.160336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.160347 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.160421 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:52.160873 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45346: BlockingRecv error: recv got EOF from 127.0.0.1:45346 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 06:37:52.419013 6742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:52.418318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45354 (local address 127.2.243.61:32863)
0807 06:37:52.418419 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.418421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.418433 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:37:52.418466 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.418467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.418479 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.418548 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:52.418939 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45354: BlockingRecv error: recv got EOF from 127.0.0.1:45354 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 06:37:52.699533 6881 catalog_manager.cc:1259] Loaded cluster ID: 0f8ca270b0064d3c81d1fa88847cd0a5
I20260807 06:37:52.699608 6881 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: loading cluster ID for follower catalog manager: success
I20260807 06:37:52.700439 6881 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: acquiring CA information for follower catalog manager: success
I20260807 06:37:52.700817 6881 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 317004fa0b444037b25d5e3b4a95d1d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:37:52.732308 6902 catalog_manager.cc:1259] Loaded cluster ID: 0f8ca270b0064d3c81d1fa88847cd0a5
I20260807 06:37:52.732396 6902 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: loading cluster ID for follower catalog manager: success
I20260807 06:37:52.733284 6902 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: acquiring CA information for follower catalog manager: success
I20260807 06:37:52.733654 6902 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e9cb94a5f37a44bc8057e1b736a85cd7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:37:52.935050 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:52.932420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.2.243.61:32863)
0807 06:37:52.932607 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.932609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.932620 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:37:52.932680 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.932681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.932732 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.932808 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:52.932970 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.933472 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.934265 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.934459 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.934541 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:52.934689 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:52.934799 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:52.934859 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:52.934901 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 06:37:52.934919 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260807 06:37:52.939976 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:52.937180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.2.243.60:45673)
0807 06:37:52.937331 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.937333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.937451 (+ 118us) server_negotiation.cc:407] Connection header received
0807 06:37:52.937648 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.937650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.937693 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.937781 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:52.937883 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.938402 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.939133 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.939370 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.939452 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:52.939633 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:52.939740 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:52.939827 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:52.939869 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 06:37:52.939889 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":57,"threads_started":1}
I20260807 06:37:52.944962 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:52.942220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41970 (local address 127.2.243.60:45673)
0807 06:37:52.942256 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.942258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.942298 (+ 40us) server_negotiation.cc:407] Connection header received
0807 06:37:52.942367 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.942368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.942378 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.942426 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:52.942583 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.943039 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.944151 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.944440 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.944537 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:52.944684 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:52.944715 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:52.944768 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:52.944864 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 06:37:52.944891 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 06:37:52.945103 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:52.941985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45378 (local address 127.2.243.61:32863)
0807 06:37:52.942024 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.942025 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.942143 (+ 118us) server_negotiation.cc:407] Connection header received
0807 06:37:52.942244 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.942245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.942258 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.942288 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:52.942469 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.943213 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.944224 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.944559 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.944596 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:52.944834 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:52.944865 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:52.944893 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:52.945010 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 06:37:52.945032 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:52.945462 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:52.941993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36896 (local address 127.2.243.62:43589)
0807 06:37:52.942241 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:52.942243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:52.942252 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:52.942296 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:52.942296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:52.942348 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:52.942438 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:52.942567 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.943338 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.944370 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:52.944709 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:52.944768 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:52.944975 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:52.945089 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:52.945320 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:52.945339 (+ 19us) server_negotiation.cc:299] Negotiation successful
0807 06:37:52.945365 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":62,"threads_started":1}
I20260807 06:37:52.947002 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6515
I20260807 06:37:52.959262 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6627
I20260807 06:37:52.971585 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 6749
2026-08-07T06:37:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3155 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:37:53 dist-test-slave-rwrg krb5kdc[7091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:37:53 dist-test-slave-rwrg krb5kdc[7091](info): set up 2 sockets
Aug 07 06:37:53 dist-test-slave-rwrg krb5kdc[7091](info): commencing operation
krb5kdc: starting...
W20260807 06:37:54.255760 3020 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/test-user.keytab.
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:37:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:37:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.243.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
Entry for principal kudu/127.2.243.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab.
I20260807 06:37:54.384358 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:38419
--webserver_interface=127.2.243.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:38419,127.2.243.61:46491,127.2.243.60:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:37:54.472239 7117 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:37:54.472532 7117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:54.472577 7117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:54.472618 7117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:54.473762 7117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:54.473847 7117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:54.473874 7117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:54.473907 7117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:54.473937 7117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:54.475466 7117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:38419,127.2.243.61:46491,127.2.243.60:33231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.62.keytab
--principal=kudu/127.2.243.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.62:38419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.243.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:54.475765 7117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:54.476017 7117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:54.481205 7130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:54.481227 7127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:54.481364 7117 server_base.cc:1034] running on GCE node
W20260807 06:37:54.481227 7126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:54.481685 7117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:54.481998 7117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:54.483264 7117 hybrid_clock.cc:648] HybridClock initialized: now 1786084674483256 us; error 38 us; skew 500 ppm
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:54.486079 7117 init.cc:377] Logged in from keytab as kudu/127.2.243.62@KRBTEST.COM (short username kudu)
I20260807 06:37:54.486521 7117 webserver.cc:460] Webserver started at http://127.2.243.62:46039/ using document root <none> and password file <none>
I20260807 06:37:54.486749 7117 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:54.486819 7117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:54.486967 7117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:54.488024 7117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data/instance:
uuid: "d392f60b9d6b4aa3862243913c13a68f"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "bb4f4a4d56ac4c4c7368d1b248655c63"
server_key_iv: "2a19fa8e2f8d7aa8660fb0564e94d045"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.488395 7117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/wal/instance:
uuid: "d392f60b9d6b4aa3862243913c13a68f"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "bb4f4a4d56ac4c4c7368d1b248655c63"
server_key_iv: "2a19fa8e2f8d7aa8660fb0564e94d045"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.489871 7117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:54.490638 7138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:54.490854 7117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:37:54.490957 7117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/wal
uuid: "d392f60b9d6b4aa3862243913c13a68f"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "bb4f4a4d56ac4c4c7368d1b248655c63"
server_key_iv: "2a19fa8e2f8d7aa8660fb0564e94d045"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.491030 7117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:54.508963 7117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:54.510924 7117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:54.511093 7117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:54.515370 7117 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.62:38419
I20260807 06:37:54.515410 7223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.62:38419 every 8 connection(s)
I20260807 06:37:54.515784 7117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/data/info.pb
I20260807 06:37:54.516861 7224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:54.518339 7224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.520219 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 7117
I20260807 06:37:54.520329 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-0/wal/instance
I20260807 06:37:54.520571 3020 external_mini_cluster.cc:1428] Setting key 916560677c8666665942fb98624f7649
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:54.528949 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.519173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.2.243.62:38419)
0807 06:37:54.519310 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.519314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.519342 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:37:54.519769 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.519771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.519787 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.519866 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.520420 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.521378 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.522630 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.523154 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.523325 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.525776 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.525784 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.525784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.525790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.527342 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.527814 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.527816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.527816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.527891 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.528162 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.528163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.528163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.528357 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.528421 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.528689 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.528793 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260807 06:37:54.529090 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.519012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:38419 (local address 127.0.0.1:41270)
0807 06:37:54.519179 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.519212 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.519641 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.519892 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.519894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.520143 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.520291 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.520295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.521481 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.521483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.522533 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.522536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.523300 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.523301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.523362 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.523888 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.523896 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.525690 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.527438 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.527447 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.527449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.527726 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.527971 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.527972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.527972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.528093 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.528443 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.528450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.528545 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.528662 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.528897 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":76,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.2.243.61@KRBTEST.COM; defaulting to no policy
I20260807 06:37:54.530097 7224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.530438 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.530324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59660)
0807 06:37:54.530377 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.530420 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 06:37:54.530601 7141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
I20260807 06:37:54.531073 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.530979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59670)
0807 06:37:54.531031 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.531056 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 06:37:54.531210 7224 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:46491: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab.
Entry for principal kudu/127.2.243.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab.
I20260807 06:37:54.547994 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:46491
--webserver_interface=127.2.243.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab
--principal=kudu/127.2.243.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:38419,127.2.243.61:46491,127.2.243.60:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:37:54.580457 7224 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } attempt: 1
I20260807 06:37:54.581074 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.580916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59686)
0807 06:37:54.580984 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.581030 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:37:54.581550 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.581425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59692)
0807 06:37:54.581475 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.581521 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 06:37:54.581662 7224 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:46491: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
W20260807 06:37:54.629635 7233 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:37:54.629851 7233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:54.629868 7233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:54.629887 7233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:54.630928 7233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:54.630963 7233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:54.630977 7233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:54.630996 7233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:54.631009 7233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:54.632467 7233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:38419,127.2.243.61:46491,127.2.243.60:33231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.61.keytab
--principal=kudu/127.2.243.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.61:46491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.243.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:54.632691 7233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:54.632859 7233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:54.637682 7242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:54.637688 7239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:54.637742 7240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:54.638211 7233 server_base.cc:1034] running on GCE node
I20260807 06:37:54.638489 7233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:54.638796 7233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:54.639972 7233 hybrid_clock.cc:648] HybridClock initialized: now 1786084674639943 us; error 60 us; skew 500 ppm
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:54.642619 7233 init.cc:377] Logged in from keytab as kudu/127.2.243.61@KRBTEST.COM (short username kudu)
I20260807 06:37:54.643078 7233 webserver.cc:460] Webserver started at http://127.2.243.61:40701/ using document root <none> and password file <none>
I20260807 06:37:54.643321 7233 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:54.643394 7233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:54.643553 7233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:54.644749 7233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data/instance:
uuid: "0529321f2a71490e83a9b8742b121e80"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "ab0f9d902cb436cfeeb4b99f01200020"
server_key_iv: "b3bc2854a71de45475a284c1e68d39f0"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.645123 7233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/wal/instance:
uuid: "0529321f2a71490e83a9b8742b121e80"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "ab0f9d902cb436cfeeb4b99f01200020"
server_key_iv: "b3bc2854a71de45475a284c1e68d39f0"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.646512 7233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:37:54.647383 7255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:54.647570 7233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:54.647684 7233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/wal
uuid: "0529321f2a71490e83a9b8742b121e80"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "ab0f9d902cb436cfeeb4b99f01200020"
server_key_iv: "b3bc2854a71de45475a284c1e68d39f0"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.647766 7233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:54.649822 7224 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } attempt: 2
I20260807 06:37:54.650269 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.650109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59696)
0807 06:37:54.650180 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.650227 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:37:54.650674 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.650569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59704)
0807 06:37:54.650620 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.650650 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 06:37:54.650805 7224 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.61:46491: Network error: Client connection negotiation failed: client connection to 127.2.243.61:46491: connect: Connection refused (error 111)
I20260807 06:37:54.657501 7233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:54.659340 7233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:54.659500 7233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:54.663843 7233 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.61:46491
I20260807 06:37:54.663904 7342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.61:46491 every 8 connection(s)
I20260807 06:37:54.664296 7233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/data/info.pb
I20260807 06:37:54.665400 7344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:54.666988 7344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.672899 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 7233
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:54.673026 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-1/wal/instance
I20260807 06:37:54.673243 3020 external_mini_cluster.cc:1428] Setting key 8125b7ba069e1ce5c49e93b52b0a2a0a
I20260807 06:37:54.676898 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.667757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.2.243.62:38419)
0807 06:37:54.667813 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.667815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.668026 (+ 211us) server_negotiation.cc:407] Connection header received
0807 06:37:54.668471 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.668471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.668487 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.668671 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.669328 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.669866 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.670854 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.671055 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.671138 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.673632 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.673634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.673634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.673641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.675371 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.675783 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.675784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.675785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.675859 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.676100 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.676101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.676101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.676256 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.676316 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.676735 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.676824 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 06:37:54.677150 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.667697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:38419 (local address 127.0.0.1:41280)
0807 06:37:54.667885 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.667928 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.668387 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.668731 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.668734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.668984 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.669181 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.669185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.669963 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.669964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.670764 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.670766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.671156 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.671157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.671274 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.671799 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.671807 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.673520 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.675465 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.675472 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.675474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.675695 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.675935 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.675935 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.675936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.676033 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.676345 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.676352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.676562 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.676686 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.676866 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260807 06:37:54.677856 7344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.243.60@KRBTEST.COM; defaulting to no policy
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.243.60@KRBTEST.COM" created.
I20260807 06:37:54.686717 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.678229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.2.243.61:46491)
0807 06:37:54.678412 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.678418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.678443 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:37:54.678467 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.678469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.678484 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.678569 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.678865 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.680068 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.680823 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.681346 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.681429 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.683513 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.683521 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.683521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.683527 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.685175 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.685652 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.685654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.685655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.685709 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.686006 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.686007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.686007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.686203 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.686313 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.686496 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.686614 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260807 06:37:54.686810 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.678160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59706)
0807 06:37:54.678220 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.678233 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.678284 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.678610 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.678611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.678727 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.678776 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.678777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.680162 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.680163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.680737 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.680739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.681434 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.681435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.681489 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.681930 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.681935 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.683435 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.685301 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.685303 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.685303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.685563 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.685841 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.685842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.685842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.685924 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.686297 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.686298 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.686356 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.686444 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.686563 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 06:37:54.687517 7344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.687919 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.687779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36236)
0807 06:37:54.687829 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.687886 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 06:37:54.688082 7256 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
I20260807 06:37:54.688620 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.688478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36250)
0807 06:37:54.688566 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.688602 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 06:37:54.688730 7344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:33231: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.243.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab.
Entry for principal kudu/127.2.243.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab.
I20260807 06:37:54.703110 3020 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTdeU2g/build/release/bin/kudu
/tmp/dist-test-taskTdeU2g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:33231
--webserver_interface=127.2.243.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab
--principal=kudu/127.2.243.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.243.20:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.243.62:38419,127.2.243.61:46491,127.2.243.60:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:37:54.737964 7344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } attempt: 1
I20260807 06:37:54.738481 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.738337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36256)
0807 06:37:54.738409 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.738450 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 06:37:54.739027 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.738884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36264)
0807 06:37:54.738961 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.738992 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 06:37:54.739156 7344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:33231: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
I20260807 06:37:54.741984 7224 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } attempt: 3
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:37:54.749840 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.742306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59714)
0807 06:37:54.742371 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.742384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.742475 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.742656 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.742657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.742839 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.742907 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.742909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.743646 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.743647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.744189 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.744190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.744666 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.744668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.744755 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.745268 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.745273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.746811 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.748681 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.748682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.748683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.748868 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.749110 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.749111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.749111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.749175 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.749489 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.749490 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.749538 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.749633 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.749737 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 06:37:54.749837 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.742386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.2.243.61:46491)
0807 06:37:54.742455 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.742458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.742470 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:37:54.742560 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.742561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.742573 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.742651 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.743038 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.743527 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.744287 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.744585 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.744667 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.746925 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.746927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.746927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.746935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.748461 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.748964 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.748965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.748966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.749031 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.749275 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.749276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.749276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.749404 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.749483 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.749660 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.749737 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:37:54.750294 7224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.750687 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.750565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36280)
0807 06:37:54.750623 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.750654 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 06:37:54.751060 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.750960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36294)
0807 06:37:54.751014 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.751038 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 06:37:54.751161 7224 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:33231: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
I20260807 06:37:54.782328 7224 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } attempt: 1
I20260807 06:37:54.782817 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.782670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36300)
0807 06:37:54.782748 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.782788 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 06:37:54.783239 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.783095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36310)
0807 06:37:54.783167 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.783211 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 06:37:54.783396 7224 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:33231: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
W20260807 06:37:54.788585 7356 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:37:54.788772 7356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:37:54.788789 7356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:37:54.788810 7356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:37:54.789903 7356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:37:54.789939 7356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:37:54.789953 7356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:37:54.789966 7356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:37:54.789978 7356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:37:54.791366 7356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.243.20:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.243.62:38419,127.2.243.61:46491,127.2.243.60:33231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/krb5kdc/kudu_127.2.243.60.keytab
--principal=kudu/127.2.243.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.243.60:33231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.243.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 05:48:25 UTC on ccycloud.quasar-xmejrm.root.comops.site
build id 4571899
I20260807 06:37:54.791612 7356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:37:54.791842 7356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:37:54.797027 7368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:37:54.796988 7369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:54.797158 7356 server_base.cc:1034] running on GCE node
W20260807 06:37:54.796988 7372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:37:54.797442 7356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:37:54.797699 7356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:37:54.798882 7356 hybrid_clock.cc:648] HybridClock initialized: now 1786084674798862 us; error 47 us; skew 500 ppm
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:37:54.801569 7356 init.cc:377] Logged in from keytab as kudu/127.2.243.60@KRBTEST.COM (short username kudu)
I20260807 06:37:54.802023 7356 webserver.cc:460] Webserver started at http://127.2.243.60:37591/ using document root <none> and password file <none>
I20260807 06:37:54.802301 7356 fs_manager.cc:359] Metadata directory not provided
I20260807 06:37:54.802374 7356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:37:54.802522 7356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:37:54.803586 7356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data/instance:
uuid: "d9b5b30db8d74542a56f2f11096b72c2"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "e13432ac25823a6f10a18e12b49318d2"
server_key_iv: "75b863d2a107bdbce91638cc6f40cb26"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.803939 7356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/wal/instance:
uuid: "d9b5b30db8d74542a56f2f11096b72c2"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "e13432ac25823a6f10a18e12b49318d2"
server_key_iv: "75b863d2a107bdbce91638cc6f40cb26"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.805339 7356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 06:37:54.806111 7381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:37:54.806306 7356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:37:54.806398 7356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data,/tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/wal
uuid: "d9b5b30db8d74542a56f2f11096b72c2"
format_stamp: "Formatted at 2026-08-07 06:37:54 on dist-test-slave-rwrg"
server_key: "e13432ac25823a6f10a18e12b49318d2"
server_key_iv: "75b863d2a107bdbce91638cc6f40cb26"
server_key_version: "encryptionkey@0"
I20260807 06:37:54.806478 7356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:37:54.807296 7344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } attempt: 2
I20260807 06:37:54.807663 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.807537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36312)
0807 06:37:54.807601 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.807639 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 06:37:54.807991 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.807895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36322)
0807 06:37:54.807941 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.807964 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 06:37:54.808086 7344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.243.60:33231: Network error: Client connection negotiation failed: client connection to 127.2.243.60:33231: connect: Connection refused (error 111)
I20260807 06:37:54.817286 7356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:37:54.819134 7356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:37:54.819307 7356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:37:54.823458 7356 rpc_server.cc:307] RPC server started. Bound to: 127.2.243.60:33231
I20260807 06:37:54.823487 7479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.243.60:33231 every 8 connection(s)
I20260807 06:37:54.823833 7356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/data/info.pb
I20260807 06:37:54.824980 7483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:37:54.826432 7483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.829026 3020 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTdeU2g/build/release/bin/kudu as pid 7356
I20260807 06:37:54.829164 3020 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTdeU2g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786084392481039-3020-0/minicluster-data/master-2/wal/instance
I20260807 06:37:54.829396 3020 external_mini_cluster.cc:1428] Setting key cb1e18860fa810453a8ba4389eb932f8
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:37:54.836967 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.827071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.62:38419 (local address 127.0.0.1:41288)
0807 06:37:54.827262 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.827302 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.827778 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.828009 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.828013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.828264 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.828448 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.828452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.829646 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.829648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.830683 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.830685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.831184 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.831185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.831311 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.831812 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.831822 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.833512 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.835293 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.835301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.835303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.835642 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.835869 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.835870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.835870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.835987 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.836257 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.836264 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.836405 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.836559 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.836727 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260807 06:37:54.837006 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.827126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.2.243.62:38419)
0807 06:37:54.827194 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.827196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.827395 (+ 199us) server_negotiation.cc:407] Connection header received
0807 06:37:54.827867 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.827868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.827877 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.827934 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.828633 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.829482 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.830784 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.831083 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.831142 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.833642 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.833643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.833644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.833652 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.835171 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.835729 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.835730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.835731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.835795 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.836071 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.836072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.836073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.836188 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.836265 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.836822 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.836899 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:37:54.837481 7483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.838824 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.829833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.2.243.60:33231)
0807 06:37:54.830033 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.830037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.830073 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:37:54.830126 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.830129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.830148 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.830227 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.830935 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.831898 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.832917 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.833283 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.833367 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.835808 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.835815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.835816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.835823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.837480 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.837956 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.837959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.837960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.838024 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.838242 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.838243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.838243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.838382 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.838537 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.838627 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.838706 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":146,"threads_started":1}
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.62@KRBTEST.COM
I20260807 06:37:54.845499 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.837847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.2.243.61:46491)
0807 06:37:54.837934 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.837936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.837951 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:37:54.837975 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.837976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.837987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.838048 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.838289 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.838749 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.839655 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.839873 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.839905 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.842073 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.842074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.842074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.842081 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.843810 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.844705 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.844707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.844707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.844765 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.844964 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.844965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.844965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.845076 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.845163 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.845372 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.845439 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:37:54.845557 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.837755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.61:46491 (local address 127.0.0.1:59716)
0807 06:37:54.837799 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.837808 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.837841 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.838062 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.838063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.838171 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.838200 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.838201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.838849 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.838850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.839527 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.839529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.839970 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.839971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.840010 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.840397 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.840402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.841963 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.844393 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.844395 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.844395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.844627 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.844834 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.844835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.844835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.844892 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.845186 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.845187 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.845228 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.845330 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.845422 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 06:37:54.846563 7483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:37:54.847784 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.839620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.2.243.62:38419)
0807 06:37:54.839664 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.839666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.839676 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:54.839696 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.839696 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.839705 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.839749 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.840261 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.840989 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.841986 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.842209 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.842257 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.844914 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.844915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.844916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.844923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.846424 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.846862 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.846864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.846864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.846941 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.847199 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.847200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.847200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.847342 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.847543 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.847592 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.847678 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.60@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.243.61@KRBTEST.COM
I20260807 06:37:54.854753 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.846826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36346)
0807 06:37:54.846907 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.846918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.846954 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.847107 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.847108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.847257 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.847288 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.847289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.848189 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.848190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.849081 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.849084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.849560 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.849561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.849620 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.850144 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.850150 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.851673 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.853474 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.853475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.853475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.853696 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.853950 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.853951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.853952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.854033 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.854343 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.854344 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.854404 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.854507 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.854606 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 06:37:54.854851 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.846919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.2.243.60:33231)
0807 06:37:54.846963 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.846966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.846991 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:37:54.847022 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.847023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.847037 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.847094 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.847366 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.848117 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.849165 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.849422 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.849560 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.851765 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.851767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.851767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.851775 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.853362 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.853796 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.853797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.853797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.853864 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.854112 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.854113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.854113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.854259 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.854346 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.854610 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.854697 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:37:54.856465 7483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: Bootstrap starting.
I20260807 06:37:54.857543 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.848640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.2.243.61:46491)
0807 06:37:54.848705 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.848706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.848716 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:54.848739 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.848740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.848750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.848820 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.849466 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.849956 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.850729 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.850955 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.851043 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.854033 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.854035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.854035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.854041 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.856189 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.856661 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.856662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.856662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.856741 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.856976 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.856977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.856977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.857083 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.857192 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.857369 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.857432 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:37:54.857724 7483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:54.858035 7483 log.cc:824] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:54.858566 7224 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } attempt: 2
I20260807 06:37:54.859169 7483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: No bootstrap required, opened a new log
I20260807 06:37:54.859505 3020 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:37:54.861562 7483 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.861725 7483 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b5b30db8d74542a56f2f11096b72c2, State: Initialized, Role: FOLLOWER
I20260807 06:37:54.861819 7483 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.860981 7182 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d392f60b9d6b4aa3862243913c13a68f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41304
I20260807 06:37:54.862180 7483 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.62@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:37:54.863618 7492 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.863739 7492 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:37:54.864629 7512 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:54.864687 7512 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:54.867170 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.858844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36350)
0807 06:37:54.858888 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.858901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.858945 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.859114 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.859115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.859286 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.859333 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.859335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.860160 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.860161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.860877 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.860880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.861304 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.861306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.861367 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.861880 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.861886 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.863554 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.865921 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.865922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.865922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.866169 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.866397 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.866397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.866397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.866460 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.866797 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.866798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.866852 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.866921 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.867051 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 06:37:54.865222 7356 master_runner.cc:186] Error getting master registration for 127.2.243.62:38419: OK, instance_id { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" instance_seqno: 1786084674514117 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:37:54.867568 7356 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:37:54.867574 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.858939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.2.243.60:33231)
0807 06:37:54.858976 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.858978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.858986 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:37:54.859018 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.859019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.859029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.859089 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.859435 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.860059 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.860987 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.861219 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.861306 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.863657 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.863658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.863659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.863667 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.865813 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.866259 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.866260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.866260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.866322 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.866531 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.866532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.866532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.866694 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.866764 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.867389 (+ 625us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.867474 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:54.869470 7224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f: Bootstrap starting.
I20260807 06:37:54.870268 7224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:54.870617 7224 log.cc:824] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:54.871615 7224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f: No bootstrap required, opened a new log
I20260807 06:37:54.873181 7224 raft_consensus.cc:348] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.873339 7224 raft_consensus.cc:729] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d392f60b9d6b4aa3862243913c13a68f, State: Initialized, Role: FOLLOWER
I20260807 06:37:54.873438 7224 consensus_queue.cc:260] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.873792 7224 sys_catalog.cc:564] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:54.873847 7182 raft_consensus.cc:482] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 06:37:54.873927 7182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:54.874897 7182 raft_consensus.cc:504] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.875151 7182 leader_election.cc:288] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [CANDIDATE]: Term 1 election: Requested vote from peers 0529321f2a71490e83a9b8742b121e80 (127.2.243.61:46491), d9b5b30db8d74542a56f2f11096b72c2 (127.2.243.60:33231)
I20260807 06:37:54.875815 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d392f60b9d6b4aa3862243913c13a68f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d9b5b30db8d74542a56f2f11096b72c2"
I20260807 06:37:54.875937 7434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:54.876987 7434 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d392f60b9d6b4aa3862243913c13a68f in term 1.
I20260807 06:37:54.877293 7139 leader_election.cc:302] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d392f60b9d6b4aa3862243913c13a68f, d9b5b30db8d74542a56f2f11096b72c2; no voters:
I20260807 06:37:54.877544 7544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:37:54.877518 7516 sys_catalog.cc:455] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.877839 7516 sys_catalog.cc:458] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:54.878036 7544 raft_consensus.cc:686] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [term 1 LEADER]: Becoming Leader. State: Replica: d392f60b9d6b4aa3862243913c13a68f, State: Running, Role: LEADER
I20260807 06:37:54.878233 7544 consensus_queue.cc:237] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.879009 7516 sys_catalog.cc:455] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d392f60b9d6b4aa3862243913c13a68f. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.879145 7516 sys_catalog.cc:458] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: This master's current role is: LEADER
W20260807 06:37:54.879307 7537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:54.879360 7537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:54.879411 7549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:37:54.879582 7549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:37:54.877667 7117 master_runner.cc:186] Error getting master registration for 127.2.243.61:46491: OK, instance_id { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" instance_seqno: 1786084674662636 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:37:54.880318 7117 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:37:54.882202 7434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [term 1 FOLLOWER]: Refusing update from remote peer d392f60b9d6b4aa3862243913c13a68f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:54.882529 7544 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:54.883924 7492 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d392f60b9d6b4aa3862243913c13a68f. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.884019 7492 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:54.886298 7516 sys_catalog.cc:455] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.886382 7516 sys_catalog.cc:458] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: This master's current role is: LEADER
I20260807 06:37:54.886562 7492 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.886650 7492 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:54.886607 7550 mvcc.cc:205] Tried to move back new op lower bound from 7315802828314025984 to 7315802828302594048. Current Snapshot: MvccSnapshot[applied={T|T < 7315802828314025984}]
I20260807 06:37:54.886795 7556 mvcc.cc:205] Tried to move back new op lower bound from 7315802828314025984 to 7315802828302594048. Current Snapshot: MvccSnapshot[applied={T|T < 7315802828314025984}]
I20260807 06:37:54.889659 7549 catalog_manager.cc:1347] Generated new cluster ID: 9761789729ee4ca482c2def1945fc777
I20260807 06:37:54.889716 7549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:37:54.899210 7344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } attempt: 3
I20260807 06:37:54.902444 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.884956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.2.243.60:33231)
0807 06:37:54.885013 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.885015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.889462 (+ 4447us) server_negotiation.cc:407] Connection header received
0807 06:37:54.889606 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.889608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.889623 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.889695 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.891548 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.892353 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.894713 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.895066 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.895119 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.898241 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.898243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.898244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.898253 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.900250 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.901113 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.901114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.901115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.901178 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.901599 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.901601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.901601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.901766 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.901854 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.902255 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.902341 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 07 06:37:54 dist-test-slave-rwrg krb5kdc[7091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786084674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.243.61@KRBTEST.COM for kudu/127.2.243.60@KRBTEST.COM
I20260807 06:37:54.910223 7549 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:37:54.911399 7549 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:37:54.911723 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.899541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.243.60:33231 (local address 127.0.0.1:36370)
0807 06:37:54.899648 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 06:37:54.899660 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:37:54.899700 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:37:54.902660 (+ 2960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:37:54.902661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:37:54.902828 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:37:54.902868 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.902870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.903801 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.903802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.904645 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:37:54.904647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.905159 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.905160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:37:54.905213 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.905662 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:37:54.905668 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:37:54.907331 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:37:54.910203 (+ 2872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.910204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.910204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.910465 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.910731 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:37:54.910732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:37:54.910733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:37:54.910802 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:37:54.911358 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:37:54.911359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:37:54.911422 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 06:37:54.911495 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 06:37:54.911610 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 06:37:54.912550 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.899791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36370 (local address 127.2.243.60:33231)
0807 06:37:54.902545 (+ 2754us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.902547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.902556 (+ 9us) server_negotiation.cc:407] Connection header received
0807 06:37:54.902575 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.902576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.902586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.902652 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.902957 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.903716 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.904730 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.905056 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.905112 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.907449 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.907450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.907451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.907459 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.910123 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.910577 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.910578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.910579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.910643 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.910905 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.910907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.910907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.911203 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.912061 (+ 858us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.912145 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.912327 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2726,"thread_start_us":93,"threads_started":1}
I20260807 06:37:54.914888 7344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: Bootstrap starting.
I20260807 06:37:54.915668 7344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: Neither blocks nor log segments found. Creating new log.
I20260807 06:37:54.915987 7344 log.cc:824] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: Log is configured to *not* fsync() on all Append() calls
I20260807 06:37:54.921234 7344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: No bootstrap required, opened a new log
I20260807 06:37:54.923470 7344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.923627 7344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0529321f2a71490e83a9b8742b121e80, State: Initialized, Role: FOLLOWER
I20260807 06:37:54.923709 7344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } }
I20260807 06:37:54.924104 7344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:37:54.924752 7300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:37:54.925725 7300 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [term 1 FOLLOWER]: Refusing update from remote peer d392f60b9d6b4aa3862243913c13a68f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:37:54.926074 7516 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:37:54.926137 7301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d392f60b9d6b4aa3862243913c13a68f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0529321f2a71490e83a9b8742b121e80"
I20260807 06:37:54.926466 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.885289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.2.243.61:46491)
0807 06:37:54.885651 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.885654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.900938 (+ 15284us) server_negotiation.cc:407] Connection header received
0807 06:37:54.900966 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.900968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.900977 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.901029 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.906258 (+ 5229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.906973 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.908544 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.908857 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.908924 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.920611 (+ 11687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.920613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.920613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.920622 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.922301 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.924883 (+ 2582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.924885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.924885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.924945 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.925144 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.925144 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.925145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.925276 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.925379 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.926269 (+ 890us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.926366 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 06:37:54.926982 7301 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d392f60b9d6b4aa3862243913c13a68f in term 1.
I20260807 06:37:54.927614 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:54.884803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41308 (local address 127.2.243.62:38419)
0807 06:37:54.884853 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.884855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.900740 (+ 15885us) server_negotiation.cc:407] Connection header received
0807 06:37:54.900822 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.900824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.900838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.900898 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.908595 (+ 7697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.909388 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.912585 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:54.912927 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:54.912970 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:54.922725 (+ 9755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:37:54.922726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:37:54.922727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:37:54.922734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:37:54.924429 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.925441 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.925442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.925443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.925515 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:37:54.925911 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:37:54.925912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:37:54.925912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:37:54.926055 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:37:54.926168 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:54.927393 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0807 06:37:54.927522 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 06:37:54.929374 7549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f: Generated new TSK 0
I20260807 06:37:54.929559 7549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:37:54.930513 7592 mvcc.cc:205] Tried to move back new op lower bound from 7315802828421074944 to 7315802828302594048. Current Snapshot: MvccSnapshot[applied={T|T < 7315802828314025984}]
W20260807 06:37:54.930711 7598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:37:54.930780 7598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:37:54.932739 7570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.932896 7570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:54.933010 7570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d392f60b9d6b4aa3862243913c13a68f. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.933147 7570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:54.933228 7570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.933415 7570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:37:54.941565 7516 sys_catalog.cc:455] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d392f60b9d6b4aa3862243913c13a68f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d392f60b9d6b4aa3862243913c13a68f" member_type: VOTER last_known_addr { host: "127.2.243.62" port: 38419 } } peers { permanent_uuid: "0529321f2a71490e83a9b8742b121e80" member_type: VOTER last_known_addr { host: "127.2.243.61" port: 46491 } } peers { permanent_uuid: "d9b5b30db8d74542a56f2f11096b72c2" member_type: VOTER last_known_addr { host: "127.2.243.60" port: 33231 } } }
I20260807 06:37:54.941668 7516 sys_catalog.cc:458] T 00000000000000000000000000000000 P d392f60b9d6b4aa3862243913c13a68f [sys.catalog]: This master's current role is: LEADER
W20260807 06:37:54.958585 7565 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.956599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.2.243.60:33231)
0807 06:37:54.956670 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.956673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.957729 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 06:37:54.957836 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.957838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.957852 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.957944 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.958477 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36376: BlockingRecv error: recv got EOF from 127.0.0.1:36376 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 06:37:54.958659 7228 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.956858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.2.243.62:38419)
0807 06:37:54.956904 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.956907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.957914 (+ 1007us) server_negotiation.cc:407] Connection header received
0807 06:37:54.958027 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.958028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.958041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.958074 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.958490 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41310: BlockingRecv error: recv got EOF from 127.0.0.1:41310 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:37:54.959344 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.957400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.2.243.61:46491)
0807 06:37:54.957441 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.957443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.958792 (+ 1349us) server_negotiation.cc:407] Connection header received
0807 06:37:54.958894 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.958896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.958907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.958938 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.959281 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59748: BlockingRecv error: recv got EOF from 127.0.0.1:59748 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 06:37:54.961902 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.961041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.2.243.61:46491)
0807 06:37:54.961080 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.961082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.961326 (+ 244us) server_negotiation.cc:407] Connection header received
0807 06:37:54.961441 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.961442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.961461 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.961509 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.961831 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59760: BlockingRecv error: recv got EOF from 127.0.0.1:59760 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 06:37:54.965116 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.964280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.2.243.61:46491)
0807 06:37:54.964316 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.964318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.964593 (+ 275us) server_negotiation.cc:407] Connection header received
0807 06:37:54.964699 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.964700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.964710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.964741 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.965070 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59770: BlockingRecv error: recv got EOF from 127.0.0.1:59770 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:37:54.969076 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.968215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.2.243.61:46491)
0807 06:37:54.968263 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.968265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.968558 (+ 293us) server_negotiation.cc:407] Connection header received
0807 06:37:54.968667 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.968668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.968678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.968709 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.969029 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59784: BlockingRecv error: recv got EOF from 127.0.0.1:59784 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 06:37:54.977653 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.976654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.2.243.61:46491)
0807 06:37:54.976693 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.976695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.977062 (+ 367us) server_negotiation.cc:407] Connection header received
0807 06:37:54.977165 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.977166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.977176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.977207 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.977607 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59800: BlockingRecv error: recv got EOF from 127.0.0.1:59800 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 06:37:54.987687 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:54.986852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.2.243.61:46491)
0807 06:37:54.986889 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:54.986891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:54.987138 (+ 247us) server_negotiation.cc:407] Connection header received
0807 06:37:54.987234 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:54.987236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:54.987245 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:54.987273 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:54.987625 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59816: BlockingRecv error: recv got EOF from 127.0.0.1:59816 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 06:37:55.006112 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:55.005196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.2.243.61:46491)
0807 06:37:55.005251 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:55.005254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:55.005446 (+ 192us) server_negotiation.cc:407] Connection header received
0807 06:37:55.005558 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:55.005559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:55.005585 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:55.005619 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:55.006057 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59818: BlockingRecv error: recv got EOF from 127.0.0.1:59818 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 06:37:55.040733 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:55.039675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.2.243.61:46491)
0807 06:37:55.039724 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:55.039726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:55.040074 (+ 348us) server_negotiation.cc:407] Connection header received
0807 06:37:55.040189 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:55.040190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:55.040202 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:55.040234 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:55.040677 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59820: BlockingRecv error: recv got EOF from 127.0.0.1:59820 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 06:37:55.107183 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:55.106175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.2.243.61:46491)
0807 06:37:55.106236 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:55.106239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:55.106397 (+ 158us) server_negotiation.cc:407] Connection header received
0807 06:37:55.106476 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:55.106477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:55.106487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:55.106543 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:55.107115 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59834: BlockingRecv error: recv got EOF from 127.0.0.1:59834 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260807 06:37:55.237279 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:55.236417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.2.243.61:46491)
0807 06:37:55.236534 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:55.236537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:55.236638 (+ 101us) server_negotiation.cc:407] Connection header received
0807 06:37:55.236739 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:55.236740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:55.236760 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:55.236833 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:55.237207 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59842: BlockingRecv error: recv got EOF from 127.0.0.1:59842 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 06:37:55.495463 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:37:55.494589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59852 (local address 127.2.243.61:46491)
0807 06:37:55.494670 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:55.494673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:55.494768 (+ 95us) server_negotiation.cc:407] Connection header received
0807 06:37:55.494875 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:55.494876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:55.494891 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:55.494986 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:37:55.495380 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59852: BlockingRecv error: recv got EOF from 127.0.0.1:59852 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:37:55.865199 7512 catalog_manager.cc:1259] Loaded cluster ID: 9761789729ee4ca482c2def1945fc777
I20260807 06:37:55.865279 7512 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: loading cluster ID for follower catalog manager: success
I20260807 06:37:55.866070 7512 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: acquiring CA information for follower catalog manager: success
I20260807 06:37:55.866333 7512 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d9b5b30db8d74542a56f2f11096b72c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:37:55.931231 7598 catalog_manager.cc:1259] Loaded cluster ID: 9761789729ee4ca482c2def1945fc777
I20260807 06:37:55.931308 7598 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: loading cluster ID for follower catalog manager: success
I20260807 06:37:55.932430 7598 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: acquiring CA information for follower catalog manager: success
I20260807 06:37:55.932821 7598 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0529321f2a71490e83a9b8742b121e80: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:37:56.011915 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:56.008846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.2.243.61:46491)
0807 06:37:56.009006 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:56.009008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:56.009067 (+ 59us) server_negotiation.cc:407] Connection header received
0807 06:37:56.009179 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:56.009180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:56.009290 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:56.009399 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:56.009588 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.010094 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.010998 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.011234 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.011315 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:56.011507 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:56.011652 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:56.011747 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:56.011789 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 06:37:56.011813 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260807 06:37:56.016314 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:56.013966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.2.243.60:33231)
0807 06:37:56.014091 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:56.014093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:56.014103 (+ 10us) server_negotiation.cc:407] Connection header received
0807 06:37:56.014138 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:56.014139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:56.014174 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:56.014247 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:56.014332 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.014798 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.015504 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.015697 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.015751 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:56.015889 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:56.016010 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:56.016078 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:56.016154 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 06:37:56.016200 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":54,"threads_started":1}
I20260807 06:37:56.021030 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:56.018090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.2.243.62:38419)
0807 06:37:56.018246 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:56.018247 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:56.018255 (+ 8us) server_negotiation.cc:407] Connection header received
0807 06:37:56.018293 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:56.018294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:56.018330 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:56.018451 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:56.018602 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.019190 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.020051 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.020261 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.020344 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:56.020541 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:56.020648 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:56.020711 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:56.020903 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 06:37:56.020930 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":54,"threads_started":1}
I20260807 06:37:56.021322 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:56.018194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.2.243.61:46491)
0807 06:37:56.018231 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:56.018232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:56.018468 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:37:56.018570 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:56.018571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:56.018582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:56.018643 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:56.018804 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.019290 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.020425 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.020720 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.020810 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:56.020966 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:56.020996 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:56.021030 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:56.021221 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 06:37:56.021244 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 06:37:56.022150 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:37:56.018138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.2.243.60:33231)
0807 06:37:56.018184 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 06:37:56.018186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 06:37:56.018468 (+ 282us) server_negotiation.cc:407] Connection header received
0807 06:37:56.018697 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:37:56.018698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:37:56.018710 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:37:56.018767 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:37:56.019551 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.020269 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.021323 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:37:56.021563 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:37:56.021600 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:37:56.021850 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:37:56.021885 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:37:56.021941 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:37:56.022042 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 06:37:56.022067 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 06:37:56.023278 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 7117
I20260807 06:37:56.037525 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 7233
I20260807 06:37:56.046783 3020 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTdeU2g/build/release/bin/kudu with pid 7356
2026-08-07T06:37:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3075 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6230 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283577 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST